Life Lessons from understanding why bestsellers sell.

When I first read the premise of The Bestseller Code by Jodie Archer and Matthew Jockers, I placed a pre-order immediately. It is something too interesting to miss given my grander ambitions.

Apparently computer scientists, through the use of Natural Language Processing (NLP) techniques which have made so much headway to programmatically de-construct a fictional bestseller, it is possible to use AI to score a piece of fiction for earning potential. Already publishers may be front-running manuscripts by passing it through an algorithm and giving only the writers with the most potential advances to reach best-seller status.

More importantly, what computer scientists can now figure out whether culture runs on principles more similar to the laws of physics.

The conclusion from this book is stunning and creates a whole new world of possibilities.

Right now, I can imagine what algorithms can already do when processing investment news. It is definitely possible to read a piece of journalistic writing and determine whether it is bullish or bearish for a piece of stock and then trigger a buy or sell call. This may be the only credible strategy when we trade crypto-currencies. At the more exotic end, I can even imagine that an AI might even be able to read a judgement from a lower court and determine whether an appeal will succeed.

I am just going to talk about two aspects of bestselling fiction which I found particularly interesting from this book which may be useful to readers like myself who are investors rather than readers of fiction :

a) Best-sellers are emotional roller coaster rides

The premise is actually very simple for a computer scientist. Simply scan the text and find sections which consists of more positive words like "love" and separate them from more negative words like "hate". Create a graph which goes up when a section is positive and goes down when a section is negative. You will create a graph which charts the emotional terrain of a piece of fictional work.

It is found that bestsellers are almost always emotional roller coaster rides with multiple highs and lows.

b) Protagonists in a bestseller must always have a high degree of agency in their lives

The second aspect of a best-seller work is that the protagonists must always have agency - lead characters must always be in control of their destiny and must do things to interact with the events and people around them. Books where the person is just a casual observer or someone who is helpless against the events around them tend to score poorer with the AI.

One way an algorithm can do this is to consider the use of verbs in bestsellers compared to those that do not sell well. For some strange reason, associating with verbs "need" and "want" always leads to bigger sales and the verb "wish".

To me this is a very teachable moment about personal development and self-help. People in general are just not excited by folks who seem to want to "wish" their personal situations to change. People who think and observe don't seem to score well with readers who want to read about folks who need or want something and so proceed to take steps to achieve their destiny.

Deep learning and AI techniques are fast invading the field of humanities which is good news for computer scientists who can find new domains to solve really tough problems which in the past would require personal judgement. Imagine the amount of work-load savings a literary agent can achieve if he can trim his pile of manuscripts by 70% but running it thorough an algorithm first. ( I think even the folks who built the algorithm admits that human judgement cannot be outsourced away at the last mile )

Imagine a piece of software that can run through an appellate brief and give an opinion as to whether there is a chance of winning a case.

This is the potential of NLP !