Using Reddit’s popular ChangeMyView community as a source of baseline data, OpenAI had previously found that 2022’s ChatGPT-3.5 was significantly less persuasive than random humans, ranking in just the 38th percentile on this measure. But that performance jumped to the 77th percentile with September’s release of the o1-mini reasoning model and up to percentiles in the high 80s for the full-fledged o1 model.
So are you smarter than a Redditor?
I wonder how many of the Reddit comments were from inauthentic sock puppets. I’d guess that subreddit was also used by influence peddlers to train and test their own human disinformation agents too.