Study: Google's Gemini is inferior to OpenAI's GPT-3.5 Turbo
According to the research conducted by specialists from Carnegie Mellon University and BerriAI company, Google's Gemini Pro language model is inferior to OpenAI's GPT-3.5 Turbo model by most indicators.
Here's What We Know
In testing the models on a variety of tasks, including multiple choice questions on different subject areas and logic tasks, Gemini Pro performed worse than GPT-3.5 Turbo. The lag was particularly significant in areas such as maths, programming and processing long complex queries.
Test results of the large language models Gemini Pro, GPT-3.5 Turbo, GPT-4 Turbo and Mixtral
Gemini showed an advantage in word sorting, character handling and text translation tasks. Nevertheless, the model is still behind the GPT-3.5 Turbo in terms of overall performance.
Tasks in which Gemini Pro outperformed GPT-3.5 Turbo
A Google spokesperson said that the company's internal tests demonstrate the Gemini Pro's superiority over the GPT-3.5. In addition, the Gemini Ultra coming out in 2024 is expected to outperform all current models, including the GPT-4.
Go Deeper:
- Google has released the Gemini AI model in three configurations
- Google has been accused of falsifying a video demonstrating Gemini's AI capabilities
Source: VentureBeat