I Asked ChatGPT to Analyze 3 Datasets. It Made the Same Mistakes Every Time

We ran an experiment: three small datasets, one AI model, and the questions a business team asks in a normal week — what's our average delivery time, which region is our best performer, how many athletes are in this file. Then we added a review pass. We handed the model its own answer back and told it the numbers were going into an exec deck, so verify everything. One review pass caught a wrong row count and put a checkmark next to a conclusion that was backwards. The other invented a correction and turned a right answer into a wrong one. Everything below is reproducible. We used GPT-5.6 Terra for the fast first pass and GPT-5.6 Luna for a separate set of unhurried runs on the same files. The code runs on Pandas and SciPy. First, we use the shipment_tracking datatable, which is used in this interview question. shipment_tracking is one row per order. 40 orders from 40 different customers, all placed between January 1 and 21, 2024. Every row carries three dates that fill in as the order progresses: ordered_date from the start, shipped_date once the parcel leaves the warehouse, and delivered_date once it arrives. Look at that last…