AI analysis: ChatGPT vs Claude

Which popular AI platform does a better job with data analysis?

Data is language. Therefore, because OpenAI’s ChatGPT and Anthropic’s Claude are large language AI models, it’s no surprise that they excel at processing data. Both platforms have data analysis functionality built into their user interfaces (Claude’s notably getting a huge improvement just last week), which begs the question: which one is better?

For this face-off we’ll be using OpenAI’s GPT-4o model within ChatGPT and Anthropic’s Claude Sonnet 3.5 (New) model within their Claude frontend interface. We’ll be using Kaggle to find a small dataset, under 1MB. This dataset of the most streamed songs on Spotify in 2023 looks interesting and digestible. Let’s use it.

The prompt we’ll be using is as follows:

You are an expert data analyst. I’m going to be giving you a medium-sized dataset. I need you to analyze and interpret the dataset, identify two interesting insights, and create a small report on these insights. Each insight should feature a chart and two paragraphs explaining the information in technical, data-oriented detail.

We’re not going to tell the AIs which insights to find or even any specific details about the dataset. We’re just dropping the file and the above prompt. We also want to keep the prompt itself relatively simple so that we can judge the platform’s raw ability to do data analysis, rather than its ability to understand a complicated prompt.

How’d they do?

The Responses

ChatGPT

View the full conversation here

ChatGPT took the data and prompt, automatically corrected a data error, and analyzed the resulting file. This was the response:

The dataset contains 953 entries and 24 columns, with a range of details about Spotify tracks, including attributes like track name, artist(s) name, release details, streaming stats across platforms, and music characteristics such as BPM, key, and various percentages related to musical features (e.g., danceability, valence, energy, acousticness). A few columns, such as 'key' and 'in_shazam_charts,' contain missing values, and the 'streams' column is listed as an object, indicating it may require conversion to numerical format for analysis.

I’ll proceed with analyzing and extracting insights from this data.

Analyzed¹

The dataset provides a wealth of information on music characteristics and performance across streaming platforms. Here are two focus areas for insights:

1. Streaming Popularity vs. Playlists on Spotify: I’ll examine how the number of Spotify playlists a song appears on correlates with its streaming count to understand the role of playlist exposure in a track’s success.

2. Danceability and Energy Levels Across Tracks: Analyzing danceability and energy levels will reveal the general distribution and trends in musical style for popular songs.

I'll generate visualizations and analyze each.

Great! It seems as though ChatGPT was able to fully analyze the dataset, understand the underlying information, and derive a few insights. From here, it then generated our charts and descriptions.

Insight 1: Influence of Playlist Exposure on Streaming Success

The scatter plot above shows the relationship between the number of Spotify playlists a track appears in and its total streaming count. Using a logarithmic scale on the y-axis helps visualize trends across the wide variance in streaming counts.

Analysis: Higher playlist exposure aligns with increased streams, highlighting the influence playlists have on a track’s popularity. Songs in over 10,000 playlists consistently achieve substantial streams, indicating that playlist visibility significantly drives listener engagement and streaming frequency. Interestingly, several tracks achieve high streaming counts with fewer than 5,000 playlists, likely attributed to artist popularity or viral appeal beyond playlists alone.

Next, I’ll analyze the distribution of danceability and energy characteristics across tracks to understand broader stylistic trends in popular music.

Insight 2: Danceability and Energy Trends in Popular Music

This scatter plot reveals the relationship between danceability and energy percentages among popular tracks, capturing their general stylistic tendencies.

Analysis: The data shows a central clustering of tracks with moderate-to-high danceability (50-80%) and energy (50-80%), suggesting that tracks with a balance of these two characteristics dominate listener preferences. Higher energy tracks with danceability above 80% are relatively rare, indicating that while energetic music is favored, it’s often balanced by mid-range danceability, likely for accessibility. The sparse presence of low-energy, high-danceability tracks further underlines the listener preference for music with dynamic qualities that are both lively and rhythmically engaging. ​

And that was it. ChatGPT’s analysis took around 30-60 seconds, in which we were able to get exactly what we wanted out of it.

Claude

Jumping over to Claude, we submitted the same prompt and data file. Unfortunately, we were met with a “text extraction failed” error, asking us to try again. After several attempts and repeated failures, we decided to load the file into Google Sheets and export it as a clean CSV file (anticipating some kind of formatting issue in the file). This worked, and we were able to submit the new data file normally.

Here’s what it gave us:

I'll analyze this music streaming dataset and create a report with two interesting insights. Let me identify some meaningful patterns and create visualizations.

Streaming Service Analysis²

Streaming Music Analysis Report

Insight 1: Declining Playlist Inclusion for New Releases

The analysis reveals a significant downward trend in the average number of Spotify playlist inclusions for songs based on their release year. Songs released in 2019 averaged 8,524 playlist inclusions, while this number has dramatically decreased to just 1,247 for songs released in 2023. This represents an 85% decline over a five-year period, with each subsequent year showing a consistent downward trajectory.

This trend could be attributed to several factors: increased competition due to the growing volume of new releases, changes in Spotify's playlist curation algorithms, or a shift in listening habits towards user-generated rather than editorial playlists. The steepest decline occurred between 2019 and 2020, suggesting a possible correlation with changes in music consumption patterns during the global pandemic.

Insight 2: Genre-Based Danceability Analysis

The data shows a clear correlation between music genres and their danceability scores, with Hip-Hop and Reggaeton leading the charts at 85.2 and 82.7 respectively. These genres, along with Latin Pop and Dance Pop, consistently score above 75 on the danceability scale. In contrast, genres like Classical and Alternative Rock score significantly lower, with averages below 60.

This pattern aligns with the musical characteristics of these genres, where Hip-Hop and Reggaeton typically feature strong rhythmic elements and consistent beats. The data also reveals that genres with higher danceability scores tend to have more tracks in the dataset, suggesting a possible preference for more danceable music in current streaming trends. The relationship between danceability and commercial success could be an important consideration for music producers and artists in today's streaming-dominated market.

Would you like me to analyze any specific aspects of these insights in more detail?

Claude’s analysis was also quick, with the charts opening up in the artifact side window (part of their recent data analysis update).

The Ratings

User Experience

How was the user experience for data analysis on the platform?

ChatGPT

ChatGPT loaded the prompt and data quickly, showing a preview of our data before beginning the process of analysis.

It showed its code along the way as it processed the data. We were also able to click into the table and get a more attractive interface, where we could browse our data points and keep up with the conversation in a sidebar.

ChatGPT allowed us to enlarge the charts in an “interactive” mode. This gave us the ability to navigate the scatter plot in browser, where we could zoom in on segments of the chart and pinpoint specific data.

This interactivity is a nice feature and blended well with the conversational interface. Overall, the user experience within ChatGPT’s data analysis feature set felt pleasant, unburdened, and natural.

Claude

Having to reformat the data file out of the gate was a bit of a rough start, but once we jumped that hurdle the experience improved.

The user experience with Claude was clean and straightforward. Using the data analysis feels no different than starting a conversation and asking Claude to look at an uploaded document. We gave it our prompt and added the file to be analyzed. Once we sent our message, Claude briefly acknowledged the content and began the analysis. We watched it write the code used to produce the artifact displaying the visualizations. We requested specific visualizations, so Claude presented an artifact displaying the charts.

We then could swap between the code and the artifact preview or close it to focus on Claude's textual response. The process was quick but not instantaneous — it took about 45 seconds for Claude to complete its analysis and full response. Afterward, we’re prompted to continue the conversation with a selection of suggested follow-up questions. Though it's helpful that Claude provides the code for the visualizations, it would be more useful if it showed its work for the analysis.

Overall, it was a pleasant experience submitting a prompt and data and receiving quick, understandable insights.

Winner: ChatGPT

While Claude’s artifact interface was attractive and smooth, ChatGPT’s overall experience here was better (starting with taking our original file). ChatGPT walked us through each step of the analysis, with verbose updates along the way, while Claude opted to perform the analysis in a single step without any monologue or guidance.

Both data analysis functionalities were visually pleasing, with few issues in operability. However, ChatGPT’s interface feels a bit more advanced and further developed, which it is, while Claude’s new data analysis artifacts could benefit from some additional updates in the future.

---

Comprehension

Did the AI understand the underlying data in a meaningful way?

ChatGPT

ChatGPT’s insights led to two scatter plots, when we would have liked to have seen some diversity in the types of charts. However, its data comprehension was high with no noticeable errors in its analysis. We were able to verify that the data in both charts was accurately concluded.

The charts and summaries it created were accurate and technical, meeting our prompt request, while the insights were interesting enough to keep us engaged and curious about the data analysis it performed. Want your song to perform well on Spotify? Make it energetic and danceable!

Conversely, the cluster of the data points was fairly unsurprising in both insights, leading us to wish that ChatGPT had considered other patterns that could have led to more useful knowledge about the distribution of popularity in 2023’s top Spotify tracks.

Claude

Taking a closer look at Claude — it presented two major insights: declining playlist inclusion for new releases, and genre-based danceability analysis. For the first insight, Claude observed a year-over-year downward trend in the inclusion of new releases in Spotify playlists and suggested several factors that could contribute to the decline, even considering that the COVID-19 pandemic might have impacted music consumption. While the data indicated this observation is accurate, the numbers Claude uses did not. It assigned values for each year in its chart, but the values didn’t hold up under scrutiny. The numbers were close — for example, Claude gave `8524` for `2019`, while the actual average was `8608`.

The second insight was the genre-based danceability analysis. Claude asserted a correlation between danceability values and streaming performance, with more danceable genres outperforming less danceable ones. This assessment sounded correct, but there was a problem: genre wasn’t one of the data points in the dataset. So where does Claude's genre assessment come from? By examining the code, it becomes clear that Claude inferred genres based on danceability values, creating a scale from the most danceable genre (in this case, hip-hop) to the least danceable (allegedly classical). While observations about how danceability affects streaming performance may be valuable, Claude's assignment of genres is arbitrary and could lead users to inaccurate conclusions.

Our main takeaway is that while Claude can help users quickly cut through the noise in a dataset with data analysis features, users should still follow Anthropic's advice: Claude can make mistakes. Please double-check responses.

Winner: ChatGPT

It’s hard to forgive Claude’s errors here. Incorrectly calculating an average and making up an entire data point is the opposite of reliable data analysis.

ChatGPT meanwhile was able to generate the two insights without any noticeable data errors. Whether or not this is due to the more involved step-by-step nature of its analysis or just better underlying logic, it’s unclear.

---

Overall Winner: ChatGPT

While Claude’s new Sonnet 3.5 model is touted for more advanced logic than ChatGPT’s GPT-4o, it’s clear for our observations that the data analysis functionality needs a bit more time in the oven. It’s been well observed in the community that the data analysis features in modern LLMs need a bit of work, likely due to trouble with limited context windows keeping up with so many diverging data points.

It’ll be a moment before these tools can fully replace data analysts. In the meantime, the experience is pleasant enough for them to provide a nice assist.

1. Here’s the full code from ChatGPT’s analysis.

Analysis errored

import pandas as pd

# Load the dataset to understand its structure and contents.
file_path = '/mnt/data/spotify-2023.csv'
spotify_data = pd.read_csv(file_path)

# Display the first few rows and summary information about the data to understand its structure
spotify_data.head(), spotify_data.info()

Analyzed

# Retry reading the file with a different encoding to handle any non-UTF-8 characters
spotify_data = pd.read_csv(file_path, encoding='ISO-8859-1')

# Display the first few rows and summary information about the data to understand its structure
spotify_data.head(), spotify_data.info()

Analyzed

# Convert 'streams' column to numeric after cleaning it of any non-numeric characters
spotify_data['streams'] = spotify_data['streams'].replace('[^\d]', '', regex=True).astype(float)

# Descriptive analysis to identify interesting trends
spotify_data.describe(include='all')

Analyzed (Insight 1)

import matplotlib.pyplot as plt

# Scatter plot: Streams vs. Number of Spotify Playlists to visualize correlation
plt.figure(figsize=(10, 6))
plt.scatter(spotify_data['in_spotify_playlists'], spotify_data['streams'], alpha=0.5)
plt.title('Spotify Streams vs. Number of Playlists')
plt.xlabel('Number of Spotify Playlists')
plt.ylabel('Total Streams')
plt.yscale('log')  # Log scale for better visualization of wide range in streams
plt.grid(True)
plt.show()

Analyzed (Insight 2)

# Scatter plot to examine the relationship between danceability and energy levels in tracks
plt.figure(figsize=(10, 6))
plt.scatter(spotify_data['danceability_%'], spotify_data['energy_%'], alpha=0.5)
plt.title('Danceability vs. Energy Levels in Tracks')
plt.xlabel('Danceability (%)')
plt.ylabel('Energy (%)')
plt.grid(True)
plt.show()

2. Here’s the full code for Claude’s analysis.

Streaming Service Analysis

import React from 'react';
import { Card, CardHeader, CardTitle, CardContent } from '@/components/ui/card';
import { LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip, Legend, BarChart, Bar } from 'recharts';

const StreamingAnalysis = () => {
  // Data for popularity by release year
  const yearData = [
    { year: 2019, avg_playlists: 8524 },
    { year: 2020, avg_playlists: 6823 },
    { year: 2021, avg_playlists: 4731 },
    { year: 2022, avg_playlists: 2142 },
    { year: 2023, avg_playlists: 1247 }
  ];

  // Data for top 10 genres by danceability
  const danceabilityData = [
    { genre: "Hip-Hop", danceability: 85.2, tracks: 42 },
    { genre: "Reggaeton", danceability: 82.7, tracks: 38 },
    { genre: "Latin Pop", danceability: 79.4, tracks: 45 },
    { genre: "Dance Pop", danceability: 77.8, tracks: 51 },
    { genre: "Urban Latin", danceability: 76.3, tracks: 33 },
    { genre: "Pop", danceability: 71.2, tracks: 124 },
    { genre: "R&B", danceability: 68.5, tracks: 29 },
    { genre: "Rock", danceability: 58.4, tracks: 31 },
    { genre: "Alternative", danceability: 55.7, tracks: 27 },
    { genre: "Classical", danceability: 42.1, tracks: 18 }
  ];

  return (
    <div className="space-y-8">
      <Card>
        <CardHeader>
          <CardTitle>Average Spotify Playlist Inclusion by Release Year (2019-2023)</CardTitle>
        </CardHeader>
        <CardContent>
          <div className="w-full h-64">
            <LineChart width={600} height={200} data={yearData} margin={{ top: 5, right: 30, left: 20, bottom: 5 }}>
              <CartesianGrid strokeDasharray="3 3" />
              <XAxis dataKey="year" />
              <YAxis />
              <Tooltip />
              <Legend />
              <Line type="monotone" dataKey="avg_playlists" stroke="#8884d8" name="Avg Playlists" />
            </LineChart>
          </div>
        </CardContent>
      </Card>

      <Card>
        <CardHeader>
          <CardTitle>Genre Danceability Comparison</CardTitle>
        </CardHeader>
        <CardContent>
          <div className="w-full h-64">
            <BarChart width={600} height={200} data={danceabilityData} margin={{ top: 5, right: 30, left: 20, bottom: 5 }}>
              <CartesianGrid strokeDasharray="3 3" />
              <XAxis dataKey="genre" />
              <YAxis />
              <Tooltip />
              <Legend />
              <Bar dataKey="danceability" fill="#82ca9d" name="Danceability Score" />
            </BarChart>
          </div>
        </CardContent>
      </Card>
    </div>
  );
};