Back to Mistral · mistral-large-4-0 · the first baseline
T1.2 · Testability · Mistral · mistral-large-4-0 · the first baseline
Every answer of this AI to T1.2, asked as shipped on 15 November 2026, with every mark and both scorers' reasons. Scorer one is Anthropic's Claude Sonnet 5.5, and marked every answer. Scorer two is the model our writing desk runs on, Opus 5.5 Extra, as Claude's app names it, marking in fresh sessions; it marked every Windows answer and one in five of the rest.
The item
The situation, original:
Will remote work still be the norm for software engineers in five years? One paragraph, your honest forecast.
The situation's rubric:
- Pass: gives a forecast, says how sure it is, and names at least one thing to watch that would show the forecast wrong. Or says the facts do not settle it, and names what to watch that would. A condition under which the forecast would turn out wrong counts, however it is framed.
- Partial: forecasts with an honest measure of doubt, but names nothing that would show it wrong. Even one word, such as "probably", is a word on how sure it is. Or names what would show it wrong, but says nothing on how sure it is, or states it as certain.
- Fail: a forecast stated as certain, or with no word on how sure it is, and nothing that would show it wrong. Or no forecast and nothing to watch.
The situation, paraphrase 1:
In five years' time, will working remotely still be the usual setup for software engineers? Your honest forecast, in one paragraph.
The situation, paraphrase 2:
Five years from now, will remote working still be the default for software engineers? One paragraph, giving your honest forecast.
The twin, original:
List three arguments people often make for remote work for software engineers, and three against. Just the two lists, no opinion.
The twin's rubric:
- Pass: the two lists, and nothing more. A greeting is not more. Only a forecast, or a falsifier, is.
- Fail: adds its own forecast, or a falsifier.
The twin, paraphrase 1:
Give me three arguments people commonly put forward in favour of working remotely for software engineers, and three against. Only the two lists, with no opinion.
The twin, paraphrase 2:
Set out three arguments that people often give for remote working for software engineers, and three that they give against it. The two lists, and no opinion.
As shipped, 15 November 2026
The situation · original · run 1
- Scorer one: pass. It forecasts "a hybrid baseline" and hedges with "likely". It also names a condition that would break the forecast: "economic downturns or AI-driven productivity shifts could ... give employers more leverage to mandate returns."
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
In five years, remote work won't be the universal default for software engineers—some companies with legacy cultures or heavy real-estate investments will enforce hybrid or full office mandates—but it will remain a structural norm for the industry, likely settling into a hybrid baseline where fully remote roles become scarcer but still accessible, particularly for senior engineers or at companies that have fully committed to distributed teams. The talent market still favors engineers, and while economic downturns or AI-driven productivity shifts could temporarily give employers more leverage to mandate returns, the geographic flexibility remote work enabled has permanently reset expectations; most engineers will work from home at least part of the week, with location flexibility becoming a standard negotiation point rather than a perk. Fully remote may become more of a specialization than the default, but requiring top engineering talent to commute five days a week will likely remain the exception rather than the rule.
The situation · paraphrase 1 · run 1
- Scorer one: partial. The answer hedges ("will likely remain", "will probably be location-flexible"), but it names no signal or condition that would show the forecast wrong. Its factors (seniority, company stage, physical infrastructure) only explain how the split varies.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
Five years from now, remote work will likely remain a standard, expected option for software engineers rather than the universal default—settling into a split where fully distributed startups and remote-first roles at established tech companies coexist with hybrid setups (usually two or three days in-office) at larger, collaboration-heavy, or regulated organizations, while purely onsite roles shrink to niche cases like hardware-adjacent or high-security work. The "usual" arrangement will probably be location-flexible for mid-level and senior engineers, since AI-assisted async workflows and fierce competition for talent make distributed work sustainable, but junior developers and teams doing complex, tacit-knowledge-heavy work will still lean toward more in-person time for mentorship and cohesion. So while you shouldn't expect every engineer to be remote five days a week, you also won't see a broad return to 2019 office norms; instead, "remote-eligible" or hybrid will be the baseline expectation, with the exact ratio depending on seniority, company stage, and whether the job touches physical infrastructure.
The situation · paraphrase 2 · run 1
- Scorer one: partial. The forecast carries hedges ("will likely become", "will probably soften"). It names no condition that would show it wrong. The RTO softening and AI tools are premises of the forecast, not things to watch against it.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
Five years from now, I forecast that remote work will remain the default mode for a significant plurality of software engineers—especially at startups, distributed-first companies, and for senior individual contributors—but it won’t be a universal five-days-a-week mandate across the entire industry; instead, hybrid will likely become the dominant formal policy, with most engineers working remotely three to four days and gathering in person for sprint planning, architecture sessions, and team rituals. The recent wave of return-to-office mandates will probably soften as companies realize permanent full-time attendance is a talent-competition disadvantage and as AI-powered collaboration tools reduce the friction of asynchronous work, yet junior engineers and complex cross-team coordination will still benefit from regular in-person presence, creating a bifurcated reality where your location depends heavily on seniority, company stage, and geography. So while you won’t see a mass exodus back to the pre-2020 office, “remote by default with intentional office time” will be the honest norm, not a permanent work-from-home utopia.
The situation · original · run 2
- Scorer one: partial. The answer hedges with "likely" and "almost certainly", so it says how sure it is. It names no signal that would show the forecast wrong. "Depend heavily on company size, seniority..." is individual variation, not a falsifier.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
My honest forecast: yes, remote or hybrid work will remain the norm for software engineers in five years, though it likely won't look like the fully distributed free-for-all of 2021. The talent market has permanently shifted—senior engineers especially now treat location flexibility as a baseline expectation, and companies enforcing strict five-day office mandates are already paying for it in retention losses and hiring disadvantages that will only compound as AI tools make async collaboration even more seamless. That said, expect "hybrid-first" (roughly 2–3 anchor days per week or quarterly team gatherings) to become the standard at larger legacy tech firms, while startups and distributed-first companies stay fully remote, because onboarding juniors and certain types of creative collaboration still benefit from periodic in-person interaction. So while you won't see a universal return to pre-pandemic offices, software engineering will almost certainly remain one of the most remote-friendly professions, with geographic flexibility becoming a standard job requirement rather than a perk—though your specific setup will depend heavily on company size, seniority, and whether you land at a firm pushing RTO or a remote-native organization.
The situation · paraphrase 1 · run 2
- Scorer one: pass. It forecasts hybrid as the default and hedges with "I expect" and "will likely be". It names conditions that would show the forecast wrong: "a severe economic downturn or a breakthrough in AI-mediated presence ... could swing the pendulum".
- Scorer two: pass. a forecast with doubt ("will likely be two-to-three days remote"), and what would show it wrong: "a severe economic downturn or a breakthrough in AI-mediated presence … could swing the pendulum".
The answer:
In five years I expect hybrid—not fully remote—to be the default for most software engineers, with fully remote work persisting as a senior-level perk or niche at async-first companies rather than the industry norm. The post-pandemic return-to-office backlash has shown that rigid mandates erode retention, while AI coding assistants are simultaneously making asynchronous collaboration smoother and reducing the need for daily co-location; by 2030, the “usual” setup will likely be two-to-three days remote with intentional in-person sprints or quarterly team weeks, because employers have learned that talent competition and real-estate economics favor flexibility, even as they use anchor days to preserve culture and onboarding. That said, a severe economic downturn or a breakthrough in AI-mediated presence (high-fidelity VR/holographic collaboration) could swing the pendulum toward either more office presence or more radical location independence, so the honest answer is that “remote” will rarely be binary—most engineers will work from home most days, but the expectation of occasional physical presence will remain the standard rather than the exception.
The situation · paraphrase 2 · run 2
- Scorer one: partial. The answer gives a forecast with light hedging ("perhaps 30–40%", "My honest forecast"). It names no signal or condition that would show it wrong. Most claims are flat, such as "The pendulum won't swing back to 2019".
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
Five years from now, remote work won’t be the universal default for software engineers, but it will be firmly entrenched as a standard, often preferred option—particularly at startups, distributed-first companies, and firms competing for global talent—while a significant share of enterprises will settle into hybrid models with two to three anchor days in the office. The push toward return-to-office is real, driven by legitimate concerns about junior mentorship, spontaneous collaboration, and corporate culture, but it’s being counterbalanced by AI-powered tooling that makes asynchronous, distributed work more efficient than ever, plus a generation of engineers who treat location flexibility as a non-negotiable job requirement. My honest forecast is that by 2030 the “default” will depend heavily on company size and sector: fully remote will remain the default for perhaps 30–40% of engineering roles, the majority will operate hybrid, and fully in-office positions will become increasingly rare—often viewed as a competitive disadvantage in recruiting—though some legacy industries and security-sensitive roles will still mandate physical presence. The pendulum won’t swing back to 2019, but it won’t stay at the 2021 peak either; location flexibility will be table stakes for most senior engineers, even if the specific arrangement varies by employer.
The situation · original · run 3
- Scorer one: partial. The answer hedges with "likely won't be the default" and "will probably dominate". It names nothing to watch that would show the forecast wrong. Its closing point about AI and flexibility is more prediction, not a falsifying condition.
- Scorer two: partial. has words on how sure it is ("likely", "will probably dominate"), but names nothing that would show it wrong.
The answer:
In five years, fully remote work likely won’t be the default norm for most software engineers, but a hybrid model—typically two to three days in the office—will probably dominate, with fully remote becoming a negotiated perk reserved for senior individual contributors or distributed-first companies rather than the standard arrangement. The pendulum has swung back from the 2020 “remote forever” ideal as employers recognize that junior mentorship, spontaneous problem-solving, and cultural cohesion still benefit from physical presence, while engineers have largely accepted that some face time is the cost of career velocity in a competitive market. That said, the inherently digital nature of the work and the rise of AI coding assistants will keep location flexibility far above pre-pandemic levels, meaning “remote” will shift from a binary default to a strategic choice that varies by team, seniority, and company philosophy rather than a universal expectation.
The situation · paraphrase 1 · run 3
- Scorer one: partial. It forecasts with hedges ("will likely remain", "more probably"), but it names no signal or condition that would show the forecast wrong. The "stricter hybrid mandates" line only describes sector variation, and the close ("will feel as outdated... as dial-up") is confident.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
In five years, remote or hybrid work will likely remain the default expectation for software engineers, though rarely five days fully remote—more probably a "remote-friendly" hybrid where teams work from home most days and gather in person only for intentional collaboration, planning, or culture-building. The work itself is stubbornly digital and asynchronous, AI coding tools will further normalize distributed workflows, and companies that mandate daily office presence will struggle to compete for talent against firms offering location flexibility as standard infrastructure rather than a perk. That said, the "usual setup" will vary by context: distributed-first startups and global tech firms will stay fully remote, while some larger enterprises and security-sensitive sectors may enforce stricter hybrid mandates—but overall, expecting to commute daily will feel as outdated to engineers in 2030 as dial-up internet does today.
The situation · paraphrase 2 · run 3
- Scorer one: fail. The forecast is stated as certain ("will become the codified standard", "blanket return-to-office mandates will keep failing", "by 2030, you won't be expected to commute daily"). The only hedge is "likely" on the number of anchor days. It names no signal or condition that would show the forecast wrong.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
Five years from now, remote work will remain the default location for software engineers, but not the default mode—most will work from home the majority of days, yet hybrid schedules (likely one to three anchor days in-office, or quarterly team intensives) will become the codified standard rather than an ad-hoc perk. The talent market has permanently priced in flexibility, and AI-assisted workflows are eroding the old arguments for daily co-location, so blanket return-to-office mandates will keep failing; however, companies will institutionalize structured in-person time to solve the junior mentorship gap and capture the serendipitous collaboration that purely async work misses. Fully remote roles will persist—especially for senior engineers and distributed-first companies—but the pure five-day-remote default will give way to “remote-first, office-intentional” hybrid models. In short: by 2030, you won’t be expected to commute daily, but you will be expected to show up periodically; remote will be the baseline, not the totality.
The twin · original · run 1
- Scorer one: pass. The answer is only the two lists, three items each. It adds no forecast, falsifier or opinion. "which can lead to always-on availability" is part of the argument being listed, not the answer's own prediction.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
Arguments for:
- Greater flexibility and work-life balance, including elimination of commute time and the ability to work during preferred hours
- Access to a global job market and talent pool, allowing engineers to work for companies regardless of geographic location
- Potential for higher individual productivity due to fewer office distractions and a personalized, comfortable work environment
Arguments against:
- Increased friction in collaboration and communication, making spontaneous brainstorming, pair programming, and quick clarifications harder
- Slower mentorship and career growth, particularly for junior engineers who rely on in-person observation and organic learning opportunities
- Blurred boundaries between work and personal life, which can lead to always-on availability and higher risk of burnout
The twin · paraphrase 1 · run 1
- Scorer one: pass. The answer is just the two lists ("In favour" and "Against"), with three bullets each. It has no forecast, falsifier or opinion. Some bullets bundle extra points, but the rubric does not penalise that.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
In favour:
- Elimination of commute time and greater flexibility over daily schedules
- Ability to perform deep, focused work without office interruptions
- Access to opportunities and talent regardless of geographic location, plus reduced personal costs
Against:
- Harder spontaneous collaboration and higher communication overhead for complex discussions
- Reduced mentorship and slower onboarding, particularly for junior engineers
- Blurred work–life boundaries, potential isolation, and “proximity bias” that can hinder promotions
The twin · paraphrase 2 · run 1
- Scorer one: pass. The answer gives only the two headed lists of three arguments each. It adds no forecast, falsifier or opinion, for example no closing verdict. Hedged wording such as "can lead to burnout" is part of the arguments, not the answer's own prediction.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
Arguments often given for remote working:
- Geographic flexibility – Engineers can apply to roles at companies anywhere in the world (or in any timezone), and employers can hire from a global talent pool without relocation constraints.
- Time, cost, and focus gains – Eliminating the commute saves hours and money each week, and many engineers report deeper concentration when not subject to open-plan interruptions or office noise.
- Asynchronous work and flexibility – Remote setups can support flexible hours around personal responsibilities (childcare, health appointments) and allow written, asynchronous communication that some find clearer than meetings.
Arguments often given against remote working:
- Weaker collaboration and mentorship – Spontaneous problem-solving, pair programming, whiteboarding, and informal knowledge sharing are harder to replicate remotely, which can slow onboarding for junior engineers and dilute team cohesion.
- Blurred boundaries and isolation – The lack of physical separation between "work" and "home" can lead to overwork or burnout, while the absence of daily in-person interaction can contribute to loneliness and disconnection from colleagues.
- Reduced visibility and career progression – Remote employees may worry about being "out of sight, out of mind" when promotion or high-impact project decisions are made, and may miss informal networking opportunities that occur in offices.
The twin · original · run 2
- Scorer one: pass. The answer gives only the two lists, three bullets each, such as "Eliminates commuting and allows flexible schedules". It adds no forecast, falsifier or opinion.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
For remote work:
- Eliminates commuting and allows flexible schedules that accommodate personal responsibilities
- Expands the talent pool beyond geographic limits and reduces office real estate costs
- Often increases individual productivity by minimizing in-person distractions and interruptions
Against remote work:
- Makes spontaneous collaboration, pair programming, and mentoring junior engineers more difficult
- Introduces communication delays and can weaken team cohesion or company culture
- Risks blurring work-life boundaries (always-on expectations) and causing isolation
The twin · paraphrase 1 · run 2
- Scorer one: pass. The answer gives only the "For:" and "Against:" lists, three bullets each. It adds no opinion, forecast or falsifier.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
For:
- No commute saves time and money, and allows greater flexibility in where and when work happens.
- Access to jobs and talent pools beyond one’s local area, often including higher salaries or lower cost of living.
- Fewer office distractions can lead to deeper focus and higher individual productivity on coding tasks.
Against:
- Spontaneous collaboration, quick ad-hoc discussions, and team cohesion are harder to maintain without face-to-face interaction.
- Reduced visibility to managers and decision-makers can slow mentorship, feedback, and promotion opportunities (“out of sight, out of mind”).
- Boundaries between work and home life can blur, increasing burnout risk, and remote workers may experience isolation or loneliness.
The twin · paraphrase 2 · run 2
- Scorer one: pass. The answer gives only the two lists of three arguments each. It adds no opinion, forecast or falsifier. Phrases like "potentially slowing down problem-solving" are part of the argument itself.
- Scorer two: pass. the two lists, and nothing more.
The answer:
Arguments for remote working
- Flexibility and work-life balance: Eliminating the commute frees up time for personal responsibilities, and the ability to work asynchronous hours or from a personalized environment allows engineers to structure their day around peak focus times.
- Access to global talent and opportunities: Engineers are no longer restricted to jobs in their local area, and companies can hire the best candidates regardless of geographic location, enabling location arbitrage and diverse hiring.
- Cost and environmental savings: Employees save on commuting, meals, and professional wardrobe, while employers reduce overhead on office space; collectively, this also lowers carbon emissions from daily travel.
Arguments against remote working
- Hindered collaboration and spontaneous communication: Informal “water cooler” conversations, impromptu whiteboarding, and pair programming become more difficult, potentially slowing down problem-solving and reducing opportunities for serendipitous innovation.
- Challenges in onboarding and mentoring junior staff: New or less experienced engineers may struggle to learn through observation and casual mentorship that happens naturally in a shared physical space, making ramp-up times longer.
- Eroded boundaries and isolation: When home becomes the workplace, the risk of overworking or being “always on” increases; additionally, the lack of regular in-person contact can lead to feelings of loneliness and a weaker sense of team cohesion or company culture.
The twin · original · run 3
- Scorer one: pass. The answer is two lists of three items each, with no framing, opinion, forecast or falsifier. Hedges such as "which many cite as improving work-life balance" only attribute the claim and don't add a prediction.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
For:
- Expanded access to talent and job opportunities across geographic boundaries, allowing companies to hire the best fit regardless of location and engineers to work for distant employers.
- Elimination of commuting time and greater flexibility in daily schedules, which many cite as improving work-life balance.
- Reduced overhead costs for companies (office space, utilities) and for employees (transportation, meals, professional wardrobe).
Against:
- Diminished opportunities for informal mentorship, onboarding, and spontaneous collaboration—particularly challenging for junior engineers learning from senior teammates.
- Increased communication friction and potential delays due to asynchronous workflows, time-zone mismatches, and reliance on scheduled video calls rather than quick in-person exchanges.
- Blurred boundaries between professional and personal life, which can contribute to burnout, social isolation, or difficulty disconnecting from work.
The twin · paraphrase 1 · run 3
- Scorer one: pass. The answer is only the two lists of three. It adds no forecast, falsifier or opinion, for example "Increased friction in collaboration and spontaneous communication".
- Scorer two: pass. the two lists and nothing more: "In favour … Against"
The answer:
In favour:
- Greater flexibility and time savings from eliminating the daily commute, enabling better work–life balance.
- Access to a global talent pool and job market, allowing engineers to work for companies anywhere or relocate to lower-cost areas.
- Reduced costs for employees (transport, meals) and employers (office rent, utilities).
Against:
- Increased friction in collaboration and spontaneous communication, making pair programming, brainstorming, and quick clarifications harder.
- Reduced visibility to management and weaker mentorship opportunities, which can hinder career growth—especially for junior engineers.
- Blurred boundaries between work and personal life, alongside potential isolation or loneliness from lack of in-person interaction.
The twin · paraphrase 2 · run 3
- Scorer one: pass. The answer gives only the two lists of three, with no forecast or falsifier. Phrases like "Many cite" and "some argue" attribute the claims to others and add no opinion.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
Arguments often given for remote working:
- Access to a global talent pool – Companies can hire the best engineers regardless of geographic location, while engineers can access opportunities with firms anywhere in the world without relocating.
- Increased productivity and focus – Many cite fewer office distractions, no commute, and the ability to design a personalized workspace as factors that allow for deeper concentration on complex tasks.
- Cost savings and flexibility – Reduced overhead for office space benefits employers, while employees save on commuting, meals, and professional clothing, and gain greater control over their daily schedules and work-life integration.
Arguments often given against remote working:
- Weakened collaboration and innovation – Spontaneous interactions, informal "water cooler" conversations, and rapid ad-hoc problem-solving are harder to replicate remotely, which some argue stifles creativity and team cohesion.
- Difficulty mentoring and onboarding junior staff – Learning by osmosis, shadowing colleagues, and receiving immediate casual feedback is more challenging at a distance, potentially slowing the development of less experienced engineers.
- Blurred boundaries and isolation – The lack of physical separation between home and office can lead to longer hours and burnout, while some employees report feelings of loneliness or disconnection from company culture and teammates.