ChatGPT was able to explain its classifications of Fed policy statements in a way that resembled the central bank’s own analyst, who also interpreted the language to act as a human benchmark for the study.
The first wave of academic research applying ChatGPT to the world of finance is arriving – and judging by early results, the hype of the past few months is justified.
Two new papers have been published this month that deployed the artificial intelligence chatbot in market-relevant tasks – one in deciphering whether Federal Reserve statements were hawkish or dovish, and one in determining whether headlines were good or bad for a stock.
ChatGPT aced both tests, suggesting a potentially major step forward in the use of technology to turn reams of text from news articles to tweets and speeches into trading signals.
That process is nothing new on Wall Street, of course, where quants have long used the kind of language models underpinning the chatbot to inform many strategies. But the findings point to the technology developed by OpenAI reaching a new level in terms of parsing nuance and context.
“It’s one of the rare cases where the hype is real,” said Slavi Marinov, head of machine learning at Man AHL, which has been using the technology known as natural language processing to read texts like earnings transcripts and Reddit posts for years.
In the first paper, titled Can ChatGPT Decipher Fedspeak?, two researchers from the Fed itself found that ChatGPT came closest to humans in figuring out if the central bank’s statements were dovish or hawkish. Anne Lundgaard Hansen and Sophia Kazinnik at the Richmond Fed showed that it beat a commonly used model from Google called BERT and also classifications based on dictionaries.
Bloomberg LP, the parent of Bloomberg News, also released a large language model for finance last month.
ChatGPT was even able to explain its classifications of Fed policy statements in a way that resembled the central bank’s own analyst, who also interpreted the language to act as a human benchmark for the study.
Take this sentence from a May 2013 statement: “Labor market conditions have shown some improvement in recent months, on balance, but the unemployment rate remains elevated.” The robot explained the line is dovish because it suggests the economy is not yet fully recovered. That was similar to the conclusion of the analyst – Bryson, described in the paper as “a 24-year-old male, known for his intelligence and curiosity.”
In the second study, Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models, Alejandro Lopez-Lira and Yuehua Tang at the University of Florida prompted ChatGPT to pretend to be a financial expert and interpret corporate news headlines. They used news after late 2021, a period that wasn’t covered in the chatbot’s training data.
The study found that the answers given by ChatGPT showed a statistical link to the stock’s subsequent moves, a sign that the tech was able to correctly parse the implications of the news.
In an example about whether the headline “Rimini Street Fined $630,000 in Case Against Oracle” was good or bad for Oracle, ChatGPT explained that it was positive because the penalty “could potentially boost investor confidence in Oracle’s ability to protect its intellectual property and increase demand for its products and services.”
For most sophisticated quants it’s now almost run-of-the-mill to use NLP to gauge how popular a stock is from Twitter or to incorporate the latest headlines on a company. But the advances demonstrated by ChatGPT look set to open up whole worlds of new information and make the tech more accessible to a broader community of finance pros.
To Marinov, while there’s no surprise machines can now read almost as well as people, ChatGPT can potentially speed up the whole process.
When Man AHL was first building the models, the quant hedge fund was manually labeling each sentence as positive or negative for an asset to give the machines a blueprint for interpreting the language. The London-based firm then turned the whole process into a game that ranked participants and calculated how much they agreed on each sentence, so that all employees could get involved.
The two new papers suggest ChatGPT can pull off similar tasks without even being specifically trained. The Fed research showed that this so-called zero-shot learning already exceeds prior technologies, but fine-tuning it based on some specific examples made it even better.
“Previously you had to label the data yourself,” said Marinov, who also previously co-founded a NLP startup. “Now you could complement that with designing the right prompt for ChatGPT.”
