Conversation

Notices

1. Jason Gorman (jasongorman@mastodon.cloud)'s status on Saturday, 11-Nov-2023 15:11:19 UTC Jason Gorman

   So, as far as I've been able to ascertain, "custom GPTs" are essentially conversations that users join half-way through. Given my experience with GPT 3.5-4, that's right around the point in the conversation where it starts to "drift". It's best at short one-to-three hit interactions. If you didn't get what you wanted by then, you ain't getting it.

   There's a sort of statistical gravity in the conversation where newer text outweighs previous text. It forgets quickly.

   • Jason Gorman (jasongorman@mastodon.cloud)'s status on Saturday, 11-Nov-2023 15:11:13 UTC Jason Gorman

     in reply to

     GPTs seem to be built on the fiction of "prompt engineering". The unfounded idea that if you start a conversation with "You are a rockstar Python programmer..." it will write better Python code. Did you think when they trained it they annotated Python code by how good it is? You think they paid armies of Python developers to go through it and rate the training data?

   • Jason Gorman (jasongorman@mastodon.cloud)'s status on Saturday, 11-Nov-2023 15:11:19 UTC Jason Gorman

     in reply to

     It's useful to remember that it doesn't "learn" as the conversation goes on. It merely has a smaller set of token paths to explore. It's a statistically constrained GPT, but the underlying model's unchanged.

     Refining the actual model can cost $millions.

     James Broadhead repeated this.