Beep Boop Beep, The Rise of Robo-Charles
As I was sitting in my plush office in Jouy-en-Josas, sipping on my artisanal latte and staring at the Bloomberg terminal, I couldn't help but wonder: is ChatGPT going to replace us all? Will the rise of artificial intelligence and automation lead to the downfall of white collar workers like myself? I decided to do some digging[1]to find out.
First of all, let's talk about ChatGPT and what it is. ChatGPT is a large language model trained by OpenAI, designed to assist users with a variety of tasks, from answering questions to generating text. It's pretty impressive stuff, but the question on everyone's mind is: will it be able to take over our jobs?
Ha, I tricked you! That wasn’t me writing; it was a robot impersonating me. It has humour, analogies, and a conversational tone—everything you have come to expect here at CPDC. It even makes jokes in the footnotes! But we are getting ahead of ourselves.
Unless you have been living in a cave for the past few weeks you have no doubt heard about OpenAI’s ChatGPT. In case you haven’t, it is a natural language chatbot that will answer your questions, produce text, write Python scripts, etc. It can also suck up hours of your time with processing times, error messages and the endless temptation to have it pitch you ideas about Christmas movies set in Riyadh or produce kindergarten level explanations of why you would want to domicile a hedge fund in the Cayman Islands.[2] The results can be quite funny but also shockingly good. The question on everyone’s mind is what the long-term impact of AI systems like this will be and to what extent they can replace humans as producers, interpreters, and synthesizers of knowledge.
One area that ChatGPT excels in is mimicking text that is written by humans, and it can be quite specific in that mimicry. To test this, I turned ChatGPT loose through the wilds of Charles Pierson Dot Com and then asked it to write new articles as me. Robo-Charles came up with gems like “Exploring the ups and downs of the bond market and why it's like your ex” and “Repo Madness: A Charles Pierson Investigation.” These articles read like a well-researched Wikipedia article at times, but the degree to which it can capture tone is pretty decent. Here is a passage from “Repo Madness:”
First things first, what is repo? Repo, short for "repurchase agreement," is essentially a short-term loan secured by collateral. Here's how it works: a borrower (usually a bank or other financial institution) will sell a security (like a Treasury bond) to a lender and agree to buy it back at a later date, usually the next day or even the same day. The lender gets to use the security as collateral and earns a little interest in the process, while the borrower gets some cash to meet its short-term funding needs. It's like pawning your grandma's antique vase to pay for your rent, except on a much larger scale and with a lot more complicated paperwork[3].
Yeah, that’s pretty much it and the fact that it kind of makes a joke using my footnote format is truly impressive. However it isn’t always that good, like this from Robo-Charles’s piece on bonds:
On one hand, bonds are a great source of stability and income for investors. They offer relatively low volatility and a steady stream of interest payments. They're the reliable partner that you can always count on. But then there's the other side of the coin. Bond prices can be volatile and unpredictable, especially when interest rates are on the rise. It's like your ex who said they would always be there for you, but then ghosted when things got tough…
But bonds also come with their fair share of risks. The biggest one is interest rate risk. When interest rates rise, bond prices tend to fall, which can result in losses for bondholders. This is because investors will often sell their bonds and buy newer, higher-yielding bonds instead. It's like when your ex starts dating someone else who is more attractive and successful than you – it hurts, but you just have to move on and find a new bond to love.
Things go off the rails here for Robo-Charles. He starts with what seems like an anodyne definition of bonds but then starts to contradict himself[4] and then it just stops making sense. When yields rise investors sell their bonds to get even higher yields which is why prices fall and that’s like when your ex finds someone hotter and richer? This is what happens with ChatGPT in a lot of instances. You get complete nonsense trash that only superficially resembles human speech. Furthermore I have found that to really get something good you must be very specific in the output you want, as well as provide it with the information your think it needs. And even then, you often have to massage the questions and know what the outcome should look like. If you put in significant effort you can produce some stunning work.
For example I was able to produce more than a millennium’s worth of Jewish thought on a made up topic. From a Torah passage it fabricated, it produced relevant Talmudic passages from the Mishnah and Gemara, successive commentaries from Maimonides and Rashi, a Kabalistic passage from the Zohar, an illustrative anecdote about the Baal Shem Tov and further commentary from the Vilna Gaon and Moses Mendelsohn giving the Mitnagged and Halskalic perspectives on this made up issue. I was even able to tie all of this together in a sermon given by a fake Reform rabbi named Sarah Levine. I knew what this should all look like and was able to guide it to produce something that was almost perfect—but it still was able to do it. Wow.
So this has been fun, but what kind of actual analysis can ChatGPT produce? For this experiment I asked ChatGPT[5] to settle in as my amanuensis and help me produce something that generally takes a lot of work to produce de novo: an equity research report. I picked a name I follow[6] so I could easily assess the quality of the final product. Let’s see if this robot really can take over. The results are interesting:
In 2021, Crocs reported full year revenues of $1.3 billion, representing a 14.6% increase from the previous year. The company also reported a net income of $109.9 million, a significant improvement from the net loss of $29.4 million in 2020. This strong performance was driven by strong demand for the company's products, as well as successful marketing campaigns and partnerships with celebrity endorsements.
So this reads really quite well, but none of the numbers are correct. Later on it says it used a DCF and comps to come up with a price target of $100 representing an upside of 20%.[7] I then asked it to explain its valuation and it used all the right words but it was just wrong. It is worth noting that it wasn’t even consistently wrong—the EPS figure it gave[8] and the P/E ratio it claimed to derive from comps[9] produced a number that was 2.5x higher than the valuation it offered. Now, in many ways these small factual errors are the easiest thing to fix. However, what we see is a being that knows what something complicated is supposed to look like but has a hard time piecing together why facts are important or indeed how they fit together. To be fair, this isn’t a specialized tool and the way this programme can produce pieces that synthesize information to produce passages explaining concepts makes me suspect that it could do a much better job if it was specifically programmed to complete this task.
However, I am less sure that it, or any artificial intelligence, can exercise the judgement necessary produce work of meaningful quality. To continue the example of an equity research report, the assembling of the data is in many ways mechanical even when a person does it but the selection and weighting of that data is much more complicated. I have no doubt that if properly set up it could produce valuations using comps, consensus projections, etc.—but I have strong doubts that it could look at years of financial statements and determine what costs are operating costs, what expenses and revenues are meaningfully recurring, and the myriad other points of art without strong human guidance at a minimum. These are decisions that are informed by rules and definitions but are executed as judgement calls on ambiguous information. While a machine is great at applying rules, I have reservations about its ability to make judgements and how it would deal with ambiguity. Furthermore, what ChatGPT excels at is taking vast amounts of existing information and putting it together—this can resemble analysis but I am not convinced it is analysis per se.
Will AI change the way white-collar, intellectual work is performed? Of course, and it already has. Will it replace us all? I doubt it.
[1] I did not actually do any digging. [ChatGPT Generated]
[2] “Domiciling your hedge fund in the Cayman Islands is kind of like storing your treasures in a beautiful castle on the beach. Your treasures will be safe and easy to access, and you can visit them whenever you want.”
[3] Don't worry, I'm not suggesting that you should actually go pawn your grandma's vase. That would be a terrible idea. [ChatGPT Generated]
[4] More than usual.
[5] In a session where I had not trained it to be me.
[6] It’s Crocs, the ugly sandal company.
[7] FYI It’s currently trading in the mid $90s
[8] Which was wrong
[9] Also wrong
