How does chat work?

A chat model does not remember a conversation. What looks like an ongoing dialogue is, on the wire, a single request containing the entire message history so far.

Each message in that list carries a role and a text body. A system message sets instructions and constraints for the whole exchange. User messages carry requests; assistant messages record what the model previously replied. Some APIs add further roles for tool calls and their results. The client assembles this array, sends it to an endpoint, and the model returns the next assistant message.

Because the API is stateless, your application owns the conversation. To continue a dialogue you send the accumulated history again, appending the latest reply; trimming, summarizing, or reordering that history is your code's job, and everything you send counts against the token budget described in the token guide.

Two practical rules follow. First, put durable instructions in the system message rather than repeating them in every user turn. Second, treat history as data you curate: stale or contradictory messages keep influencing the model until you remove them.