LLMs and APIs discussion

• What happens when we ask an LLM a question?
• Does an LLM understand a question in the same way a human does?
• What does it mean to say that an LLM predicts the next token?
• Is an LLM simply a database containing everything it has learned?

• What is a token?
• Is one token always one word?
• Why does token count matter to a developer?
• What is the context window?
• Does an LLM remember everything from a conversation?
• What happens when the context becomes too large?

Compare:

Explain recursion.

with:

Explain recursion to a first-semester
computer science student using a simple
Java example. Maximum 100 words.

Discuss:

• Why might these produce different answers?
• What makes a prompt useful?
• Is the prompt part of our application’s logic?
• How can constraints and context improve a response?

Question:

If I call GET /products/42 twice, what do I expect?

Versus:

If I ask an LLM the same question twice, what do I expect?

Discuss:

• Why might an LLM give different answers to the same question?
• When is variation useful?
• When could variation be a problem?

• Why might an LLM confidently give an incorrect answer?
• Does fluent language mean the information is correct?
• What is a hallucination?
• How could an application detect invalid output?

Consider:

{
  "price": -500,
  "quantity": 999999999
}

The JSON is valid and Jackson can deserialize it.

But:

• Is it valid application data?
• Should we store it?
• Who is responsible for validating it?

Compare:

The capital of Denmark is Copenhagen.

with:

{
  "capital": "Copenhagen",
  "country": "Denmark"
}

Discuss:

• Which response is easier for Java to work with?
• Can the JSON be converted into a DTO?
• What happens if a property is missing?
• What if the JSON structure is correct but the information is wrong?

public record CountryInfo(
    String capital,
    String country
) {}

Consider these tasks:

• Calculate VAT
• Translate a paragraph
• Sort a list of numbers
• Summarize an article
• Validate an email address
• Generate quiz questions
• Calculate 17 * 31
• Classify customer feedback

For each task: Would you use an LLM? Why or why not?