← Applied AI for Accounting WorkflowsModule 1 of 6

Foundations: What Large Language Models Are and How to Think About Them in a Firm

You’re reading this module without an account, and the whole course is free to read this way. Sign in and enroll to record your progress and take the assessment for a certificate of completion.

Module 1 — Foundations: What Large Language Models Are and How to Think About Them in a Firm

Why this module exists

Most accountants meet artificial intelligence through a chat box. You type a question, text comes back, and it either looks brilliant or looks like nonsense. Neither reaction tells you much about what actually happened, and neither prepares you to build the technology into a firm where accuracy, confidentiality, and professional responsibility are not optional. The purpose of this module is to replace the magic-box mental model with a working understanding — enough that you can predict when a tool will help, when it will mislead you, and when you should not reach for it at all.

We will not cover the mathematics of neural networks. You do not need them, any more than you need to understand the internals of a spreadsheet's calculation engine to build a reliable workpaper. What you need is an accurate intuition for how these systems behave, because their behavior is different from every other piece of software you have used. A traditional application is deterministic: the same input produces the same output, every time, and a wrong answer is a bug someone can fix. A large language model is probabilistic: the same input can produce different output, and a wrong answer is often not a bug at all but the system working exactly as designed. If you carry the deterministic mental model into this work, you will trust the tool in precisely the situations where it is least trustworthy.

What a large language model actually is

A large language model, or LLM, is a program trained to predict the next unit of text given the text that came before it. That is the whole mechanism. During training, the model was shown an enormous quantity of written material and adjusted billions of internal parameters until it became very good at one narrow task: given a sequence of words, estimate what word is likely to come next. Everything you have seen an LLM do — draft an email, summarize a document, propose journal entries, explain a tax concept — is that single next-word prediction, run over and over, each predicted word fed back in so the next one can be predicted.

This sounds far too simple to produce the fluent, apparently reasoned output these tools generate. The surprising empirical fact of the last several years is that it is not too simple. When you train a next-word predictor at sufficient scale on sufficiently varied material, the statistical patterns it must internalize to predict well start to look a great deal like knowledge and reasoning. To predict the next word in a paragraph about depreciation, the model must have absorbed something about how depreciation is discussed. To continue a sentence that began "The recovery period for qualified improvement property is," it must have encoded the association between that phrase and "15 years." The model does not know this the way you know it. It has no concept of the Internal Revenue Code, no model of a building, no awareness that a client exists. It has a statistical representation of how such things are written about.

That distinction is the single most important idea in this entire course, so it is worth stating plainly. An LLM predicts language that is likely; it does not retrieve facts that are true. Most of the time, likely language and true facts overlap, because the training material was mostly written by people trying to be accurate. But the overlap is a tendency, not a guarantee, and the model has no internal mechanism that distinguishes a true statement from a plausible-sounding false one. Both are just text, and both can be equally likely as a continuation.

Tokens and the context window

Two pieces of vocabulary will make everything else easier to reason about.

The first is the token. The model does not process whole words; it processes tokens, which are chunks of text roughly three-quarters of a word long on average. "Depreciation" might be one token or split into two; a number like "1,743,811.07" is several. This matters for two practical reasons. Cost is measured in tokens, so a longer document literally costs more to process. And the model's memory is measured in tokens, which brings us to the second term.

The context window is the amount of text the model can consider at one time — everything you have typed in the current conversation, everything it has replied, and any documents you have pasted or attached, all counted together in tokens. Think of it as the model's working desk. A modern model might have a context window measured in hundreds of thousands of tokens, which sounds limitless, but two properties make it a real constraint. First, it is finite: paste a 400-page document into a small window and the beginning falls off the desk before the model reaches the end. Second, and less obviously, models attend unevenly across the window — information at the very start and very end of a long context tends to be used more reliably than information buried in the middle. A relevant clause you placed in paragraph 200 of a 500-paragraph prompt may be treated as if it were not there.

The practical consequence for firm work: the context window is not persistent memory. When you start a new conversation, the model remembers nothing from the last one. It has no ongoing knowledge of your clients, your prior instructions, or corrections you made yesterday, unless you supply that information again in the current window. A model that gave you a perfect trial-balance summary this morning starts this afternoon's session as a blank slate. This is a feature for confidentiality — nothing lingers by default — and a discipline for workflow design, because anything you need the model to "know" must be present in the prompt.

Why the same question gives different answers

If you ask an LLM the same question twice, you will often get two differently worded answers, and sometimes two substantively different ones. This unsettles accountants, and it should, because we work in a domain where the same inputs must produce the same outputs. The variation is deliberate, and understanding its source tells you how to control it.

At each step, the model does not simply pick the single most likely next token. It produces a probability distribution across many possible next tokens and then samples from that distribution. A setting called temperature governs how much randomness enters the sampling. At a temperature near zero, the model almost always takes the most probable token, and its output becomes nearly repeatable — the same prompt yields close to the same answer. At higher temperatures, it more readily picks lower-probability tokens, producing more varied, more "creative" output and, generally, more errors. Most consumer chat interfaces run at a moderate temperature you cannot see or change; many professional tools and programming interfaces let you set it explicitly.

For accounting work, the lesson is directional and simple. When you want consistency, structure, and adherence to a format — reconciliations, classifications, extractions, anything where there is a right answer — you want low temperature. When you want brainstorming, alternative phrasings for a client memo, or a range of ideas, higher temperature is fine. But note the deeper point: even at temperature zero, an LLM is not guaranteed correct. Low temperature buys you consistency, not accuracy. A model can be perfectly repeatable and repeatably wrong.

Hallucination: the defining failure mode

The industry's term for an LLM stating something false with complete fluency and confidence is hallucination, and it is the failure mode you must design every firm workflow around. A hallucination is not a glitch or a rare corner case. It is the direct, expected consequence of a system that generates likely-sounding language without a mechanism for checking truth. When the model does not have a strong statistical signal for the real answer — because the fact is obscure, or recent, or specific to your client, or simply was not well represented in its training — it does not stop or say "I don't know." It generates the most plausible-looking continuation, which may be entirely fabricated and will be delivered in exactly the same confident register as a correct answer.

The examples that matter most to accountants are the ones that look most authoritative:

Notice what is common to all three: the output looks more trustworthy, not less, precisely when it is invented, because fluent authority is what the model is best at producing. This is why "it sounded right" is worthless as a verification standard, and why the human review step is not a nicety we recommend but the load-bearing element of every workflow in this course.

Capabilities you can rely on, and limits you cannot design away

Set against that failure mode, LLMs have genuine, reliable strengths. The trick is to match the work to the strength.

They are strong at transformation — taking information you provide and restating, reformatting, summarizing, or reorganizing it. Turning a wall of meeting notes into a structured action list, rewriting a terse internal note as a client-ready paragraph, converting a description into a first-draft spreadsheet formula: these lean on the model's core competence with language, and the source material is right there in the window to check against. They are strong at drafting, giving you a competent first version of an email, memo, or engagement letter that you then edit — faster than a blank page, and the editing is where your judgment lives. They are strong at explanation and ideation, walking through how a concept generally works or listing considerations you might have missed, useful as a thinking partner rather than an authority.

Their limits are the mirror image. They are unreliable at recall of specific facts — exact figures, real citations, current rates — because that is retrieval of truth, not generation of likely language. They are unreliable at arithmetic and precise calculation; a language model is not a calculator, and while some tools now route math to an actual calculator behind the scenes, you cannot assume that and must never trust an LLM's raw arithmetic on a workpaper. They have a knowledge cutoff — training ended on some past date, so recent law changes, this year's rates, and anything after that date are outside what they learned, and asking anyway invites a confident hallucination. And they have no access to your specifics unless you provide them: no view of your client files, your firm's positions, or the actual document unless it is in the window.

A note on newer model types: reasoning models and multimodal input

The field moves quickly, and two developments are worth understanding because they change what the tools can do without changing the fundamentals you just learned.

The first is the emergence of reasoning models — models specifically built to work through a problem in a series of internal steps before answering, rather than generating an answer in a single pass. In practice these models "think" for longer, sometimes visibly, and they tend to do better on tasks with real logical structure: multi-step calculations, problems where an intermediate error compounds, questions that require holding several constraints at once. For accounting work this is genuinely useful, because much of what we do is multi-step. But note carefully what has and has not changed. A reasoning model still generates language, still has a knowledge cutoff, and still hallucinates — it can now reason its way to a wrong answer more elaborately, and the elaborate chain can make a wrong conclusion look more trustworthy, not less. Reasoning models raise the ceiling on what the tool can attempt; they do not remove the need to verify, and they do not turn a language model into a source of truth. They also cost more and run slower, which ties back to the model-selection theme: use them where the reasoning earns its keep, not for reformatting a list.

The second is multimodal input — models that accept images, PDFs, and sometimes audio alongside text. This matters enormously for accountants, because so much of our source material arrives as a scanned invoice, a photographed receipt, a PDF pay application, or a spreadsheet image. A multimodal model can read a document image directly rather than requiring you to transcribe it, which is the entry point for several workflows later in this course. The same cautions apply with extra force: a model reading a low-quality scan can misread a figure, transpose a digit, or confidently report a number that a smudge obscured. The convenience of "just show it the document" does not relax the requirement to check what it extracted against the actual document. If anything, extraction from images deserves more scrutiny, not less, because the failure — a misread digit stated as fact — is exactly the kind of quiet, plausible error that a workpaper cannot survive.

Neither development changes the mental model. A reasoning model is a next-token predictor that generates more intermediate tokens before the answer; a multimodal model is a next-token predictor that can take pixels as well as text as input. Both are more capable and both fail in the same ways. Understanding them as extensions of the same machine, rather than as a different kind of thing, is what keeps your judgment calibrated as the tools improve.

A working rule for when not to use AI

Put the strengths and limits together and a decision rule falls out. Use an LLM when the source of truth is in front of it and your job is to transform, draft, or triage; be far more careful — or decline — when the model must supply facts from its own memory, perform exact calculation, or reach a conclusion that will not be independently checked.

Concretely, do not reach for a consumer AI tool when:

Notice that none of these say "never." They say "match the tool to the task, and keep a professional in the loop." That is the posture the rest of the course builds on.

The human-in-the-loop principle, stated once so it can be assumed everywhere

Every workflow you will learn shares one non-negotiable structure: the AI produces a draft or a triage, and a qualified professional reviews, corrects, and takes responsibility for the result before it reaches a client or a filing. The model is an assistant that accelerates the first ninety percent; the CPA owns the last ten percent, which is where the judgment, the accuracy, and the professional responsibility all live. This is not a limitation to be engineered away as the technology improves. It is the correct division of labor between a fluent-but-unreliable text generator and a licensed professional accountable for the work. We will restate this in specific terms in Module 2, but from here forward, assume it under every prompt in this course: the output is an input to your judgment, never a substitute for it.

Anchor example: where cost segregation fits

Throughout this course, cost segregation serves as the recurring worked example, because it concentrates the whole lesson into one workflow. A cost segregation study reclassifies portions of a building's cost into shorter depreciable lives — five, seven, and fifteen years — instead of the default long recovery period, accelerating depreciation deductions. The analysis begins with unstructured, messy source material: contractor pay applications, invoices, asset listings, closing statements, a trial balance. Someone has to read all of it, sort components into recovery-period buckets, and reconcile the pieces against the total.

That reading, sorting, and first-pass triage is exactly the kind of language transformation an LLM does well, and we will build a detailed workflow for it. But — and this is the point that makes cost segregation the ideal anchor — the AI's output is strictly preliminary. A study reclassifying property for a real tax filing requires a full engineering-based cost segregation study performed by qualified professionals. The AI reads the asset list and proposes buckets for a CPA to verify; it drafts the client data request; it flags what a real study still needs. It does not produce the study, and its classifications are not filed. Hold that shape in mind: the AI does the fast, language-heavy first pass on material that is right in front of it, and the professional does the analysis, the verification, and the work that carries a name and a license. That is the model for everything ahead.

Recap

An LLM is a next-token predictor that generates likely language, not retrieved truth. It thinks in tokens and remembers only what is in its finite context window, which is not persistent memory. Its output varies because it samples from a probability distribution, tunable through temperature — lower for consistency, though consistency is not accuracy. Its defining failure is hallucination: fluent, confident fabrication, most dangerous exactly when it looks most authoritative, and especially in citations, specific figures, and clean-but-wrong reasoning. It is reliable for transformation, drafting, explanation, and triage of material placed in front of it; unreliable for factual recall, exact arithmetic, current facts past its knowledge cutoff, and anything relied upon without review. The safe posture is to match the tool to the task and keep a qualified professional in the loop on every output. In the next module, we make the confidentiality and professional-responsibility side of that posture rigorous.

Next

Disclaimer: The information provided on this platform is for general informational purposes only and does not constitute tax, financial, legal, or investment advice. Cost segregation studies and depreciation benefits vary based on property type, ownership structure, and applicable federal and state tax law. Results are estimates only. You should consult a qualified tax professional, CPA, or attorney before making any tax-related decisions. ClickDrag Finance does not guarantee specific tax outcomes.