Blog

When you interact with a large language model (LLM), you’re not chatting with an all-knowing mind. You’re engaging with a system that has a very real memory limit. That limit is called a context window. It's the box that decides how much the model can ‘remember’ and work with at any one time.If your conversation stays inside the box, you get coherent, accurate responses. If you spill over, the model starts to lose track of earlier points. That's when replies can get shaky.What Is a Context Window, Really?Think of it like this: you're carrying a small notepad. You can write down key points from a meeting, but the pad only has a few pages. If the meeting is short, you can keep track of everything. If it drags on, you’ll have to start erasing earlier notes to make room for new ones. Same with LLMs. The context window is their ‘notepad’.Once it’s full, the model can't see the older parts anymore. It starts relying on guesswork rather than actual memory. That’s why keeping conversations or inputs within the window matters.Tokens: The Units That Fill the WindowNow, what fills up the context window isn't words, exactly. It's tokens. A token can be a whole word, part of a word, or even a few words combined.For example:"Running" might be broken into "run" and "ning.""The" is usually one token."Artificial intelligence" might sometimes be tokenised as two or three units depending on the model.A rough guide: 100 English words = about 150 tokens.The tool that chops text into tokens is called a tokenizer. Different models have slightly different ways of slicing up language, but the principle stays the same.How Big Is a Context Window?Context window sizes have exploded. Early models like GPT-2 had around 2,000 tokens. Modern heavyweights like Claude 3 and IBM Granite 3 are pushing 100,000+ tokens.Sounds massive, but it's not just your conversation filling the space. System prompts, documents, code snippets, instructions — they all count. It doesn’t take much to chew through 128,000 tokens if you’re attaching reports, API data, or long conversations.Why Attention MattersHere’s where it gets interesting.LLMs use something called a self-attention mechanism to process tokens. Rather than reading left to right like we do, self-attention allows the model to look across the whole input and figure out what parts are important to each other.Imagine trying to follow a novel. You don’t just remember the last sentence; you tie together characters, earlier events, and clues from different chapters. That’s what self-attention lets LLMs do — at lightning speed.The catch? The more tokens you feed it, the more relationships the model has to map. And mapping every token against every other token gets very expensive, very quickly.The Hard Truth About Long Context WindowsBigger context windows aren't always better.Here’s why:Quadratic Costs: Doubling the number of tokens doesn’t just double the compute needed. It quadruples it. That means slower speeds, bigger server costs, and higher failure rates.Performance Drops: A 2023 study showed models perform best when important info is at the start or end of the input. Bury key details halfway through a massive input, and the model struggles — just like you'd struggle to find one note buried on page 400 of a 1,000-page book.Security Risks: Longer contexts mean more room for trouble. Malicious instructions could hide deep inside a long input, bypassing safety filters. The bigger the window, the harder it is to catch everything.The Balancing ActThere’s no perfect answer. It’s about finding a workable middle ground:<ul><li>Enough room to handle real conversations or documents.</li><li>Small enough to stay fast, affordable, and secure.</li><li>Structured well so key information isn’t buried.</li><li>Good design — not just size — makes the difference.</li></ul>Why This Matters to YouIf you’re building tools with LLMs — chatbots, code assistants, document analyzers — you need to understand context windows.Push past the limit, and the model starts forgetting. Stay inside it, and you get coherent, smart outputs. Knowing where that line is makes everything work better.When you hear about "128k context" models, remember: it’s not a magic wand. It’s a bigger box. And how you use the space inside it is what really matters.FAQs on LLM Context WindowsWhat is a context window? It’s the model’s short-term memory. It defines how much conversation or input the model can ‘see’ at once. Overfill it, and earlier parts are lost.What is a token? A token is the basic chunk of text an LLM processes. Could be a character, a piece of a word, a whole word, or a short phrase.How big are today’s context windows? Leading models can handle up to 128,000 tokens. But that space gets eaten up fast by prompts, documents, and instructions.Why not make context windows infinitely big? Because the computing power needed scales exponentially. Bigger contexts also risk performance drops and make models easier to hack.How does attention fit in? Self-attention is how models map relationships between all tokens. It's what lets them "understand" language rather than just regurgitating it.Is bigger always better? No. Good design, smart input management, and context awareness beat brute force every time.Want to go deeper? Join our <a data-mce-href="https://aistrategyconsulting.com.au/introduction-to-ai-5-day-bootcamp" href="https://aistrategyconsulting.com.au/introduction-to-ai-5-day-bootcamp" data-mce-selected="inline-boundary">5 Day AI Bootcamp </a>where we go through advanced prompting techniques like chain of thought prompting and context setting with your own assets uploaded.&nbsp;

Why AI Models Forget Things - The Simple Truth About Context Windows

When I started testing no-code AI tools for the businesses I work with, I wasn’t looking for flashy features—I just wanted something practical. Something that could cut down meeting fatigue, streamline training, and maybe even handle the basic support queries without needing three tools and five browser tabs open.That’s when I tried a platform that changed how I think about communication inside a team: a multimodal AI tool that works with voice, video, chat, and screen sharing—all in one place. It’s called Gemini Stream, and if your business is even mildly remote or customer-facing, this kind of tool is going to make a difference.No integrations. No coding. Just one interface that lets people communicate naturally—with an AI that actually gets it.So, what makes this worth your time?If your team’s juggling Teams Messages, Slack messages, Zoom meetings, Teams Meetings, OneNote docs, Notion docs, and a customer support or relationship platform, you already know how scattered communication can get. And when you try to drop AI into the mix? That usually means hiring developers or stitching together custom workflows.Gemini Stream works differently. It gives you real-time interaction using text, voice, video, or screen sharing—and the AI processes all of it together. That’s what "multimodal" means. You’re not locked into typing prompts. You can literally talk through a problem, share your screen, and the AI keeps up like an actual collaborator.The kicker? You don’t need to code a thing to start using it.How I’m seeing businesses use it already:Over the last few months, I’ve been helping teams plug tools like Gemini Stream into their workflows. Here’s where I’ve seen it genuinely work—not just as a gimmick, but as a solid improvement to how teams function:1. Support teams that don’t sound robotic Instead of relying on templated replies, reps can screen share with customers, walk through problems, and get smart AI suggestions on the fly. No more asking someone to describe an error in an email when you could just see the issue together.2. Internal training that actually sticks I’ve watched HR leads use it for onboarding—running a training video, letting new hires ask spoken questions mid-way, and guiding them through tasks live with AI support. It’s miles ahead of a clunky learning portal and a PDF.3. Faster decision-making in meetings Because the AI can summarise key points in real-time and catch follow-ups you’d normally miss, meetings are tighter. You walk away with action items, not just vague next steps.4. Live demos with real-time backup Sales teams are using it to run product walkthroughs where the AI watches the customer’s screen and feeds suggestions to the rep mid-call. It’s like having a silent technical expert sitting beside you the whole time.Why no-code and low-code actually matter hereThis isn’t some enterprise platform that takes six months to implement. I’ve helped founders and ops teams get set up in a week. That’s because tools like Gemini Stream are built with no-code and low-code users in mind. You can customise workflows, create templates, and build smart automations—all without touching a line of code.What that means practically:<ul><li>No waiting on dev resources</li><li>No expensive software engineers</li><li>No excuses not to start</li></ul>It’s also scalable. Start with just your support team, then roll it out to sales, training, or leadership calls.If you’re thinking about trying AI in your business, start hereI’ve tested dozens of tools over the last 12 months—some overhyped, some genuinely helpful. Gemini Stream is one of the few that made me stop and rethink how teams could work together more naturally, even when they’re remote or async.If you’re already stretched thin trying to build better workflows or scale your customer service, this kind of tool gives you a shortcut.And if you’re not technical? Even better. This is built for people like you—ops managers, founders, trainers, marketers—anyone who needs AI to just work.Want help rolling out something like this?I work with businesses that want to put AI to work—without hiring developers or blowing budgets. Whether you're looking to:<ul><li>Upgrade your customer support</li><li>Run more efficient internal meetings</li><li>Automate your training and onboarding</li></ul>We can help you implement no-code AI tools that actually save time and improve how your team communicates.Get in touch if you want to see how something like Gemini Stream fits into your workflow. I can walk you through how w’ve set it up for other businesses—and what it would look like for yours: <a href="https://aistrategyconsulting.com.au/ai-no-code-low-code-implementation" data-mce-href="https://aistrategyconsulting.com.au/ai-no-code-low-code-implementation">AI Low Code/No Code Implementation</a>

The AI Tool That’s Quietly Fixing the Way Teams Communicate

Female alpacas on a farm in Central Oregon

In the past year, nearly every business leader I’ve spoken to has tried ChatGPT, Gemini, or Copilot. Most are still experimenting. A few have made it stick. But a quiet shift is happening in the background—especially among firms with sensitive data, tight budgets, or a need for speed.They’re building their own AI.No, not from scratch. They’re using open-source models like Llama 3.1 or Nemotron, running them locally on secure infrastructure, and tailoring them to specific tasks. The goal? Keep costs low, control data, and get results faster than the cloud allows.Here’s why that matters—and what I’ve seen firsthand.Why local AI is gaining traction Most people assume AI means plugging into OpenAI or Microsoft. But that comes with trade-offs:<ul><li>Data risk: Even with protections, many organisations don’t want sensitive material leaving their network.</li><li>Performance: Cloud-based tools can be slow, especially under load or with complex chains of reasoning.</li><li>Cost: The more your team uses commercial AI tools, the higher the bill. These costs creep quickly.</li></ul>Running a local model sidesteps all three.With today’s open-source alternatives—like Llama 3.1, Mistral, or Nemotron—you can now fine-tune, deploy, and operate a model on-premises or on dedicated cloud resources. These models are smaller, cheaper to run, and increasingly competitive in quality.And because they run locally, you get:<ul><li>Full control over the data and model</li><li>Faster response times with no API bottlenecks</li><li>Fixed infrastructure costs instead of usage-based surprises</li></ul>That’s especially appealing to businesses in legal, healthcare, defence, finance, or research—anywhere privacy and performance aren’t negotiable.What this looks like in practiceI worked with one client who wanted to automate complex research queries across thousands of academic papers. Initially, they used GPT-4. It worked—but it was slow, costly, and required constant prompt tweaking.We switched to a fine-tuned local model running Llama 3.1 with a domain-specific retrieval chain. Not only did it speed up processing by 40%, but it also brought hosting costs down by 70%. The team could iterate faster, with full transparency on what the model was doing and why.We’re seeing similar results across legal case search, compliance workflows, internal knowledge assistants, and even frontline agent support tools.You don’t need a Data Science PhD—or a dev teamHere’s the part most people miss: this kind of deployment is now accessible to non-technical teams.With modern no-code and low-code platforms—like LangChain, Flowise, or Dust—you can wire up a local reasoning model, plug it into your data, and deploy a custom tool in days, not months. Many of the hardest parts (like embeddings, vector search, or prompt orchestration) are now prebuilt into visual workflows.You still need to understand the logic behind the model. But you don’t need to build infrastructure from scratch or write thousands of lines of Python.So what’s the catch?Like most thing AI - it is nunaced. Local models aren’t right for everything. They’re smaller, so they might not match GPT-4 on abstract creative tasks or general language fluency. But for reasoning within a known domain, they’re more than capable—and far more efficient.The challenge is knowing where they fit in your business, how to tune them for your use case, and how to get your team using them properly.What to do nextIf you’re wondering how to reduce your AI spend, protect sensitive workflows, or speed up internal tools—this is the next step.We help service businesses implement no-code and low-code AI solutions, including:<ul><li>Setting up local and open-source AI models securely</li><li>Building custom tools that align with your workflows</li><li>Training teams to manage and iterate without needing developers</li></ul>If that’s something you want to explore, I’ll show you exactly where it fits—and how to test it, fast.→<a data-mce-href="https://aistrategyconsulting.com.au/ai-no-code-low-code-implementation" href="https://aistrategyconsulting.com.au/ai-no-code-low-code-implementation" data-mce-selected="inline-boundary"> Book a No Code/Low Code AI Discovery Call</a>We’ll find the low-hanging fruit and show you how to go from experiment to impact—without overbuilding or overspending.

Why Some Businesses Are Skipping ChatGPT—and Building Their Own Local AI Instead

Microsoft Windows family tablet and smart phone

Most people don’t wake up thinking they need an AI assistant. But they do wake up thinking they need more time, fewer emails, and fewer tabs open by the end of the day.That’s where Microsoft Copilot quietly slips in—not as a headline-grabber, but as a practical, task-focused sidekick. And for many teams, it’s already changing the rhythm of the workday without anyone needing to become a tech expert to use it.The pitch is simple: what if the tools you already use—Word, Excel, Outlook, Teams—had a built-in assistant that could help summarise, rewrite, find, generate, and automate in just a few clicks?It’s no longer a what-if. It’s here.What makes Copilot different from just asking ChatGPT a question?The biggest difference is integration. Copilot sits inside the apps people already use, not alongside them. It means you don’t need to open a separate chat window and copy-paste information around. You can ask it to rewrite a report in Word, generate a table in Excel, or summarise a long Teams thread—all in context, without switching tools.The second difference is memory. Because it’s plugged into your documents, calendars, emails, and calls (with your permission), it can work more like a real assistant. It remembers what was said in meetings. It knows your deadlines. It understands the formatting of your proposals. And it can suggest or generate work that feels tailored—because it is.A day with Copilot, in practiceImagine this: You’ve got a team meeting in 20 minutes and haven’t read the notes from the last one. Copilot can summarise them.Your inbox has 40 unread messages. Copilot can prioritise them, pull out action items, and draft replies.You’re putting together a monthly report. Copilot can scan the Excel data, highlight trends, and draft a plain-English summary you can copy straight into PowerPoint.None of these tasks on their own are ground-breaking. But done together—without bouncing between apps, and without doing the heavy lifting yourself—they save hours. That’s the point.Who’s this actually useful for?At the surface level, Copilot looks like a productivity boost for individuals. But the real power kicks in when teams begin to adopt it together. Shared knowledge becomes more accessible. Repeated tasks get faster. Briefs become clearer. Documentation becomes searchable. And meetings become more focused—because everyone’s actually read the summary this time.And for businesses that don’t have in-house tech teams or AI experts, it means you can get the benefits of AI without building anything from scratch.That’s where no-code and low-code platforms matter.Bring it into your business—without the complexityWe work with service-led organisations—particularly in hospitality, tourism, and travel services—to take tools like Microsoft Copilot and make them part of everyday workflows.We help you:<ul><li>Identify where it can be used best (hint: not everywhere),</li><li>Set up automation around it using low-code tools,</li><li>Train your team on what to do (and what not to do),</li><li>And measure the time you get back as a result.</li></ul>You don’t need to overhaul your systems. You just need a practical plan—and a partner who knows what to focus on.Quickly integrate AI into your business using low-code platforms. Our approach simplifies the process, making it accessible and efficient for companies of all sizes—ensuring swift implementation without technical complexity.

Microsoft Copilot: The AI Assistant That’s Quietly Changing How We Work

There’s been a lot of noise lately about AI agents doing everything from scheduling meetings to running full workflows across different tools. Scroll through LinkedIn and you’ll see folks claiming their agents are managing entire departments. That all sounds cool—until you try asking one to do something simple like order a coffee and watch it fumble.That disconnect made me curious. Why do AI agents seem so powerful on paper, yet stumble in practice? After getting into the weeds of how they work, it turns out a lot of the hype comes from a fundamental misunderstanding of what these agents actually are. Spoiler: a lot of what people are calling “agents” are just fancy automations.Here’s what’s really going on.What AI Agents Can Actually Do TodayAI agents aren’t useless—but their strength isn’t in the showy stuff you see online. They shine in places where language meets data. Think: extracting information, transforming formats, and doing repeatable tasks inside a well-defined sandbox.For example, something like: “Go into this transcript, find all the names, and plug them into Salesforce.” That works. Not just kind of, but with surprising accuracy.But when you try to get agents to behave like independent assistants—navigating your computer, hopping across tools, completing whole tasks—they fall short. And that’s not because no one’s figured it out yet. It’s because the software systems we use today aren’t designed to interact with AI like that. APIs are built for humans and developers, not autonomous agents. So, agents hit walls. Fast.That said, some platforms are starting to build the missing pieces. But full, seamless autonomy? That’s still years off.The 3 Levels of AI Agent Maturity (And Where We’re Actually At)It helps to think about agent progress in stages. Right now, we’re firmly in stage one. Here’s how I break it down:Level 1: Works reliably (this is what we’ve got now) These agents:Handle specific, narrow tasks.Operate inside a very clear set of rules.Are great at transforming data between formats or tools. Example: Pull emails from a form and drop them into a spreadsheet.Level 2: Emerging but hit-or-miss These agents are:Just starting to interface with everyday software tools.Able to do basic coordination between apps (kind of).Built with slightly smarter commands like “create a calendar event” rather than just clicking buttons. They work sometimes. But not always. And when they don’t, you usually have to go back and fix it manually.Level 3: Pure hype for nowThis is the world people wish we lived in:Fully autonomous agents.Ones that browse the web, understand UI, click around like a human.Smart coworkers you hand work off to. Sounds dreamy. But we’re not there. And we won’t be for a while.So when someone says they’ve got an AI that runs their business? They’ve probably just hooked up a bunch of pre-set prompts and workflows.The Real Opportunity: Less Magic, More PredictabilityAI agents are messy because they’re built on probability. Give the same prompt to a model twice, and you might get two completely different results. That’s why asking an agent to follow a complex, multi-step prompt is like playing roulette. Sometimes it works. Sometimes it crashes and burns.One fix is to feed agents really detailed instructions to cover every possible scenario—but that gets out of hand fast. It’s hard enough getting one human to follow a long list of directions, let alone an unpredictable system.A smarter workaround is to have the agent generate code that handles the task. Once you’ve got code that works, it’ll keep doing the same thing every time—no surprises.That’s how some platforms are approaching it now. You ask the agent to perform a task. It writes the code. You check it. Then it runs that code the same way each time. It’s not sexy, but it’s stable. And stable beats fancy any day of the week.What the Future Really Looks Like (Without the Hype)So, where’s this all headed? Here’s the most realistic take I’ve heard—from someone deep in the agent-building world:We’ll all become AI managers. Agents won’t replace us. They’ll need us. Think of it more like managing an intern. You’ll still need to give direction, tweak results, and check their work.Predictable will beat autonomous. The agents that succeed won’t be the ones that can do anything. They’ll be the ones that do one or two things really well, the same way every time.Full autonomy is still research-grade. That idea of having a digital coworker? The kind that clicks around your screen, answers emails, and books meetings without help? It’s still mostly a concept. Not a product.The real power is in integration. Agents are most valuable when they become the thread tying all your tools together. Instead of manually moving data between apps, the agent connects the dots and keeps things in sync behind the scenes.So yes, AI agents are cool. But not because they’re futuristic sidekicks or robot colleagues. They’re cool because they offer a glimpse into what’s coming—and remind us how far we still have to go.Until then, I’ll take an agent that can update a spreadsheet reliably over one that might do something impressive... if I’m lucky!

AI Agents Sound Flashy—But Here's What's Actually Going On

What Is Agentic AI, and How Will It Change Work?There’s a lot of noise around AI right now—but one of the concepts that's starting to get real traction is “agentic AI.” If you haven’t come across that term yet, you're not alone. Until recently, it sat on the fringes of most business conversations. But with some big moves in AI development this past year, agentic systems are suddenly on everyone’s radar—and for good reason.These aren’t just another round of smart tools. Agentic AI goes far beyond predictive models or chat-based assistants. These are autonomous, goal-oriented systems that can take initiative, make decisions, and coordinate tasks with minimal human direction. That shift—away from tools we use, toward agents that act—changes everything about how we think of work, teams, and even the skills we’ll need going forward.So, what does that actually mean for the workplace?Here’s a breakdown of where agentic AI is headed, and how it’s likely to impact how we work.Most people are used to AI that reacts. You give it a prompt, and it gives you a result. Agentic AI doesn’t wait around for instructions. It operates more like a junior colleague: you give it a goal, and it works out how to achieve it.<blockquote>Agentic systems combine planning, reasoning, and memory with the ability to take multiple actions across platforms. Some are even designed to learn from outcomes and adapt. Think of them less like a tool, and more like an autonomous project manager or analyst.</blockquote>The major shift here is agency—hence the name. Agentic AI doesn't just respond. It acts.How This Will Reshape Jobs: When AI agents take on multi-step tasks without constant supervision, the way we structure work has to evolve. Here's what’s likely to change:Task ownership will move: Routine coordination tasks—like chasing up reports, sending reminders, or compiling research—are exactly the kind of work agentic AI will take off your plate. That frees up people for higher-value thinking, but it also means fewer roles based around task execution.More emphasis on direction, less on doing: The value in many jobs will come from setting objectives and reviewing outcomes, not completing each step manually. Roles that once required ten steps may now require two: decide what needs doing, and check what comes back.A rethink of ‘team’: As AI agents join the workforce, you may find yourself working with—or managing—non-human team members. That comes with new coordination challenges, from how you assign work to how you track progress.Greater demand for soft skills: Ironically, as machines get better at tasks, human skills like creativity, judgment, empathy, and collaboration become more important. These are still tough for machines to replicate, which makes them more valuable in hybrid human-AI teams.New Skills on the Radar: Just like the spreadsheet changed accounting or email changed communication, agentic AI is going to reshape which skills matter most.Expect to see rising demand for:<ul><li>Prompt engineering – Knowing how to brief AI agents effectively is a must. The quality of the output depends entirely on the clarity and structure of the input.</li><li>Workflow design – It’s not just about what gets done, but how. Knowing how to structure processes where humans and agents work side-by-side is a real edge.</li><li>AI literacy – Understanding the capabilities and limitations of AI agents will be key, especially for managers. You need to know when to trust an agent and when to step in.</li><li>Change leadership – With so much of work shifting, helping teams adapt is just as important as any technical skill.</li></ul>It’s about understanding what’s possible, what’s appropriate, and how to work productively in a new kind of workplace.What To Do Now If you’re leading a team, running a business, or just trying to stay ahead of where the market’s going, here are some next steps worth considering:Start small with agentic pilots Try using autonomous agents on internal projects—research, customer follow-ups, or data collation. Focus on low-risk areas where experimentation won’t disrupt your core operations.Upskill your team: Bring in workshops that break down AI strategy in a way that’s practical and hands-on. Teams need to learn with the tools, not just about them.Redesign processes for co-working: Start thinking of AI agents as part of your team. Which tasks can they take over? How will you monitor their output? What does escalation look like if something goes wrong?Get strategic about AI governance: As agents get more autonomous, ethical use and oversight become even more critical. Establish clear policies on how decisions are made, who’s accountable, and where AI fits into your broader strategy.We’re entering a phase where AI doesn’t just support your work—it shares it. That’s a big shift. But like most tech shifts, the winners won’t be the ones who resist change. They’ll be the ones who understand it early and start planning accordingly.Agentic AI isn’t science fiction. It’s already here—and it’s changing the shape of modern work. The smart move? Get ahead of it now.If you're building an AI strategy, this is the moment to go deeper. Whether through a full AI Strategy Blueprint or a workshop tailored to your business, the time to learn is before the curve catches up.

AI Blog

Why AI Models Forget Things - The Simple Truth About Cont...

The AI Tool That’s Quietly Fixing the Way Teams Communicate

Why Some Businesses Are Skipping ChatGPT—and Building The...

Microsoft Copilot: The AI Assistant That’s Quietly Changi...

AI Agents Sound Flashy—But Here's What's Actually Going On

What is Agentic AI and How Does it Work?