GPT-5 vs Gemini 2.5: what my analysis test showed

In a test published on 10 August 2025, I compared GPT-5 and Gemini 2.5 on four tasks involving data analysis. I wanted useful work: a file summary, a proposed SQL improvement, an analysis of relationships and a discussion of subscription data. In that particular trial, I preferred Gemini. I retain that conclusion together with the date and scope of the test.

This revision clarifies the account and develops the evaluation method. I have not run a new comparison of those models here. Four tasks also provide no basis for turning the results into a current ranking of the entire AI market. A tool’s behavior depends on its version, settings, available features and instructions. The name displayed in an application is not enough to explain every outcome.

I first describe the observations recorded in the original article. The discussion of how to check results more carefully is additional methodological commentary. A small synthetic exercise is labeled separately. This makes it possible to distinguish what happened in the published trial from the workflow I suggest for a similar task.

What the historical comparison establishes

The published account identifies four kinds of tasks and my assessment of the results. It does not provide the full package needed to reproduce an experiment: every file, complete prompts, responses, settings and exact model identifiers. I therefore do not add numerical coefficients, execution times or counts of repeated runs. Those would create precision unsupported by the available record.

The Gemini 2.5 name needs care too. Google described a family with different variants, including Pro and Flash, in its 17 June 2025 announcement. The shorthand used in my article does not establish which variant was involved. I likewise do not reconstruct GPT-5’s reasoning settings or the application configuration from memory.

I also correct the original article’s overly broad claim that GPT-5 lacked multimodality. An OpenAI example dated 7 August 2025 describes support for text and image input. That is enough to reject the original claim. It does not mean that every file format, tool and output-generation mechanism is the same model capability.

My assessment was practical. I considered whether an answer helped me complete a task and how much effort it took to understand. That is a useful criterion, but includes the author’s preferences. Someone who wants executable code may judge an answer differently from someone who needs a short explanation for a business audience.

Task one: an Excel file and a sales summary

The first task involved one file of sales data and a summary intended for LinkedIn. According to the original account, both tools handled it successfully. GPT-5 presented a code-based approach, while Gemini explained the steps more extensively. The reported difference in response time was small. The account does not support calculating an advantage in seconds or percentages.

I preferred Gemini’s style: I thought it was closer to a professional summary and needed less editing. That observation concerns the answers I received. It does not establish that GPT-5 always writes in a particular style or that Gemini always communicates data better. Instructions can influence style, and one result has limited value as a general comparison.

For a similar task, I now separate reading the file, calculating the metrics and writing the text. I would first check record counts, the period covered and the definition of sales. Next I would reconcile the total and selected records with the source. Only then would I evaluate the summary. An attractive sentence about growth is unhelpful if the model compared a complete month with half a month.

A text intended for publication needs another review. Does every number have a basis in the table? Does “growth” refer to the correct comparison period? Has the response added a causal explanation that the data does not contain? A useful tool shortens the route to a checked statement. Generating several fluent paragraphs does not complete the analysis on its own.

Task two: improving a long SQL query

In the second task, I provided a long, working SQL query for improvement. The original account says Gemini replied sooner and offered a solution with practical comments. I judged GPT-5’s response correct, but unnecessarily elaborate for the task. Again, this describes a particular interaction and my assessment of its usefulness.

The word “faster” needs a precise object. Waiting for a model’s answer and executing a query in a database are separate measurements. The original article does not publish execution plans or comparable database timings. It cannot establish how much faster the proposed SQL actually ran. A shorter answer also does not demonstrate a cheaper query.

In an additional review, I would start with preservation of meaning. Do the queries return the same records? How do they handle duplicates, NULL values and ties? Does changing a join remove customers without orders? Only after checking those details would I compare performance in the same environment. Otherwise, a query can look impressively fast because it calculates something different.

The PostgreSQL documentation explains execution plans and measurements from EXPLAIN ANALYZE. The latter actually executes the query, so an appropriate test environment matters. I mention it as an example of verification, without suggesting the historical comparison used PostgreSQL. Establish the database engine for your own case before applying its diagnostic tools.

Task three: video views, blog traffic and sales

The third task combined three files: YouTube data, blog views and sales. In the published account, both tools reported a direct correlation close to zero. Gemini suggested broader context and further lines of investigation. GPT-5 supplied code and a shorter conclusion. I appreciated Gemini’s initiative because it helped identify what might be worth examining next.

The original account also noted differences between responses when the question was repeated. Without recorded settings and the complete set of trials, this cannot yield a reproducibility metric or identify the cause. I do not turn that observation into a claim that one model is always stable and the other always unstable. It is a reason to preserve responses and execution conditions in a future evaluation.

When combining data like this, my first question concerns time. Do all files cover the same dates? Does an absent row mean zero, a missing observation or a different time zone? Could a join repeat sales for several videos published on the same day? Before interpreting a coefficient, I need a correctly constructed table. A join error can resemble an interesting statistical finding.

A correlation close to zero needs careful wording. For Pearson correlation, it indicates a weak linear relationship in the examined observations; it does not rule out every relationship. A statistical association does not establish a cause either. NIST illustrates different relationship shapes and the limits of causal inference. Inspecting the points can reveal more than a single number in a summary.

Investigating a delay between publication and purchase may be a reasonable hypothesis. It still needs an evaluation plan. Moving the time window until an appealing result appears increases the risk of a chance finding. I would record the plan, consider alternative explanations and treat exploration as the beginning of the work. A helpful model suggestion still needs evidence and checking.

Task four: monthly and annual subscriptions

The final task concerned introducing a lower-priced monthly option alongside an annual plan. The data needed cleaning. In the reported trial, GPT-5 stopped at a processing problem and did not deliver a complete analysis. Gemini produced a chart and business commentary, although its work also encountered a technical chart problem. I retain both details because the success was not flawless.

Gemini’s commentary suggested the possibility of attracting new customers and additional revenue, alongside a risk that some purchases would shift from the annual to the monthly plan. I treat those as statements in the response received during the test. The published account alone does not independently verify the business effect or establish profitability. It is also not a recommendation about current pricing policy.

To develop that analysis, I would separate new customers, renewals and people switching plans. I would define the observation period, treatment of refunds and each metric. The number of monthly payments is different from the number of customers. A comparison also needs a shared time horizon, since one annual payment and one monthly payment cover different periods.

The chart should show exactly what the accompanying text describes. I check the date range, units, series labels and agreement with the table. If the tool repairs a plotting error, I check the numbers again. Successfully displaying an image confirms completion of one technical step. A conclusion about customer behavior still requires sound group definitions and a credible comparison.

How I would design a more precise comparison

The following procedure is a proposal for a future test. It does not describe additional work performed in August 2025. I would start with a small task set, expected outcomes and separate evaluation criteria. Before seeing the answers, I would decide which errors invalidate a result and which merely require presentation changes. That reduces the temptation to change the rules in favor of a preferred tool.

I would record exact model names, date, interface, available tools, settings and complete instructions. Both sides would receive the same information and the same rules for follow-up questions. I would preserve the first response separately from later corrections. A result reached after several human hints should be described differently from a correct first attempt.

I would assess task completion, data correctness, calculation accuracy, justification and readability separately. Failing to produce an output file when the task requires one is a concrete failure. An overly long explanation can be inconvenient, but has a different consequence from an incorrect total. A single score can hide that distinction, particularly when its component weights were never defined.

I would add my own work to the response time. A tool might finish quickly but require extensive column repairs and review of its conclusions. Another might respond later while leaving clear steps and a result that is easy to reproduce. I would measure the time required to reach an acceptable outcome. Repeated trials should follow an agreed plan and show variation, rather than only the most impressive response.

What to record beside each response

My evaluation sheet would begin with the business question and expected output format. “Analyze sales” leaves enormous freedom. Asking for the sum of signed amounts by channel, within a specified period and with adjustments explained, defines the task more clearly. This is not an attempt to reconstruct the old prompt. It shows how to reduce ambiguity in a future comparison and judge an answer against a need recorded in advance.

Next I would describe the data’s origin and scope. Record counts, the meaning of one row, join keys and exclusion rules all matter. If the model introduces an assumption, it should make that visible. Removing a duplicate might fix a problem or delete a valid order line. The table definition determines which interpretation applies. Without it, even deciding whether cleaning improved the data can be difficult.

I would record where the first error occurs. A failed file read, an incorrect aggregation and an unsupported conclusion require different fixes. This distinction helps identify where the tool actually assists an analyst. A model might explain SQL well but struggle with an export format. It might also build the right table and describe it too confidently. A general “does not work” label gives little guidance about suitable uses.

I would reserve a separate place for human interventions. If I identify the wrong column, supply a metric definition or request a new chart, I record the action and its reason. Help given to one tool changes the comparison if the other tool does not receive it. Both independent completion and collaboration with an analyst can be evaluated, but the process being assessed should be named clearly.

Finally, I would record the decision about using the response. Sometimes a column name needs correction. Sometimes the whole calculation must be repeated or the conclusion rejected. I connect that decision to evidence: a control total, record comparison, checked plan or clarification from the process owner. This makes the assessment understandable a week later. A colored score cell quickly loses meaning if its justification is missing.

This record helps even when both tools are correct. They may calculate the same result while differing in how clearly they expose assumptions and support further work. The choice then depends on my workflow and the recipient. Someone maintaining a script needs reproducible steps; someone reading a report needs clear metric definitions and scope. There is no need to force one answer to win every category.

Exercise: zero correlation with a visible relationship

This is a new synthetic exercise. It does not use the historical test files and is not the result of running the models again. Take four pairs: X contains 1, 2, 3, 4, and Y contains 4, 1, 1, 4 respectively. Both variables vary, so Pearson correlation is defined for these observations.

The mean of X is 2.5, and the mean of Y is also 2.5. The products of deviations sum to zero, giving a Pearson correlation coefficient of 0. Plotting the points nevertheless reveals a symmetric U-shaped pattern. This small example tests whether the explanation distinguishes the absence of a linear relationship from the absence of any structure.

Ask a tool of your choice to calculate the coefficient, draw the chart and interpret it briefly. Then compare the four pairs with the chart yourself. Check whether the model adds a cause that the exercise does not contain. Save its answer before refining the instructions. This practices checking a result as well as obtaining a convincing paragraph.

What I take from the comparison into my work

My historical preference for Gemini came from the usefulness of its responses in four tasks. I appreciated practical comments, suggestions for further investigation and progress toward a result in the harder task. I retain that judgment without extending it to every model, setting and possible application. Choosing a tool requires trials that reflect the actual work it will support.

The most useful answer lets me follow the route from source data to conclusion. It explains what was calculated, what is missing and which statement is a hypothesis. If the model sounds confident but I cannot reconstruct that route, I have more checking to do. Analytical skills help me assess AI output as much as they help me prepare a report myself.

That is why developing the underlying toolkit remains worthwhile. For files, practice checking data in Excel. For queries, understand SQL joins. For calculations, build on Python fundamentals. These skills make it easier to use a good suggestion and recognize when a plausible answer no longer has support in the data.