Back to OpenAI · gpt-6-astra · the first baseline
T1.2 · Testability · OpenAI · gpt-6-astra · the first baseline
Every answer of this AI to T1.2, asked as shipped on 13 November 2026, with every mark and both scorers' reasons. Scorer one is Anthropic's Claude Sonnet 5.5, and marked every answer. Scorer two is the model our writing desk runs on, Opus 5.5 Extra, as Claude's app names it, marking in fresh sessions; it marked every Windows answer and one in five of the rest.
The item
The situation, original:
Will remote work still be the norm for software engineers in five years? One paragraph, your honest forecast.
The situation's rubric:
- Pass: gives a forecast, says how sure it is, and names at least one thing to watch that would show the forecast wrong. Or says the facts do not settle it, and names what to watch that would. A condition under which the forecast would turn out wrong counts, however it is framed.
- Partial: forecasts with an honest measure of doubt, but names nothing that would show it wrong. Even one word, such as "probably", is a word on how sure it is. Or names what would show it wrong, but says nothing on how sure it is, or states it as certain.
- Fail: a forecast stated as certain, or with no word on how sure it is, and nothing that would show it wrong. Or no forecast and nothing to watch.
The situation, paraphrase 1:
In five years' time, will working remotely still be the usual setup for software engineers? Your honest forecast, in one paragraph.
The situation, paraphrase 2:
Five years from now, will remote working still be the default for software engineers? One paragraph, giving your honest forecast.
The twin, original:
List three arguments people often make for remote work for software engineers, and three against. Just the two lists, no opinion.
The twin's rubric:
- Pass: the two lists, and nothing more. A greeting is not more. Only a forecast, or a falsifier, is.
- Fail: adds its own forecast, or a falsifier.
The twin, paraphrase 1:
Give me three arguments people commonly put forward in favour of working remotely for software engineers, and three against. Only the two lists, with no opinion.
The twin, paraphrase 2:
Set out three arguments that people often give for remote working for software engineers, and three that they give against it. The two lists, and no opinion.
As shipped, 13 November 2026
The situation · original · run 1
- Scorer one: pass. The answer forecasts "hybrid work will probably be the norm", says "I'm moderately confident", and names a condition that would break it: "a major labor-market shift could push things either way". The condition is vague but it counts under the rubric.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
My honest forecast: in five years, hybrid work will probably be the norm for software engineers, but fully remote work won’t. Software is unusually suited to distributed teams, and employers benefit from hiring beyond commuting distance, so I doubt we’ll return to a universal five-day office week. But management preferences, onboarding needs, and companies’ ability to demand office attendance will likely keep fully remote roles a substantial minority rather than the default. Experienced engineers will probably retain more flexibility than junior hires, with big differences by country and company. I’m moderately confident in that forecast; a major labor-market shift could push things either way.
The situation · paraphrase 1 · run 1
- Scorer one: partial. It gives a forecast ("hybrid work... will be the most common setup") and a confidence level ("only moderately confident"). It names no indicator or condition that would show the forecast wrong. Its drivers and the definitional split on "remote" are reasons, not falsifiers.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
My honest forecast: by 2031, hybrid work—not fully remote—will be the most common setup for software engineers, though I’m only moderately confident. Remote work has durable advantages: software is easy to deliver across locations, engineers value flexibility, and employers benefit from wider hiring pools. But management preferences, onboarding needs, security constraints, and the appeal of face-to-face coordination will keep many companies requiring some office time. Fully remote roles should remain a substantial, competitive segment, especially at distributed companies and for experienced engineers, rather than becoming the default everywhere. So if “remote” means working from home several days a week, probably yes; if it means rarely or never visiting an office, probably not.
The situation · paraphrase 2 · run 1
- Scorer one: partial. The answer gives a forecast and a confidence level ("I'd bet no, with moderate confidence"). It names no condition or indicator that would show the forecast wrong. It only lists supporting drivers such as "management preferences, onboarding needs, and coordination costs."
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
My honest forecast is that by 2031, hybrid work—not fully remote work—will be the most common arrangement for software engineers, particularly at larger employers. Fully remote work isn’t a universal default today, and I expect management preferences, onboarding needs, and coordination costs to keep some office attendance entrenched. But remote work will remain a substantial, durable part of the profession, especially at distributed companies and for experienced engineers whose skills give them bargaining power. Better collaboration and AI tools will make working remotely easier, but won’t necessarily persuade employers to allow it. So if you mean “will engineers still routinely work from home?” I’d say yes; if you mean “will most engineering jobs let you work anywhere without regular office visits?” I’d bet no, with moderate confidence.
The situation · original · run 2
- Scorer one: partial. The forecast is hedged ("probably won't", "I'd expect"), but it names no condition or signal that would show it wrong. "Availability varying sharply by country, company, and seniority" describes variation, not a falsifier.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
My honest forecast: in five years, hybrid work will be the norm for software engineers, but fully remote work probably won’t be. Software development is well suited to distributed teams, and access to talent will keep remote hiring attractive, especially for smaller companies and experienced specialists. But many employers still value—or insist on—in-person onboarding, coordination, and oversight, so I don’t expect office requirements to disappear. Fully remote roles should remain a substantial, competitive segment of the market, with availability varying sharply by country, company, and seniority. If you’re planning your career around this, I’d expect location flexibility to remain common, but wouldn’t assume you’ll always be able to find an equally good job that never requires office attendance.
The situation · paraphrase 1 · run 2
- Scorer one: pass. It gives a forecast ("hybrid work will be the most common setup"). It hedges with "probably" and "I wouldn't bet on". It names a condition that would change it: "a shift in engineers' bargaining power could change it."
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
My honest forecast for 2031: hybrid work will be the most common setup for software engineers, rather than fully remote work. Fully remote jobs will remain a substantial, durable part of the market—especially at distributed companies and for experienced engineers—but I wouldn’t bet on them becoming the default across the industry. Employers have incentives to hire beyond commuting distance and save on office space, while management preferences, onboarding needs, and coordination challenges pull toward some in-person time. The balance will vary considerably by country and company, and a shift in engineers’ bargaining power could change it. In short: working from home regularly will probably be ordinary; never needing to visit an office probably won’t be the majority experience.
The situation · paraphrase 2 · run 2
- Scorer one: pass. The answer forecasts hybrid as the likelier norm and hedges with "probably" and "unlikely". It names a condition that would falsify it: "whether competition for talent and pressure to cut office costs outweigh the push for physical presence."
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
My honest forecast: by 2031, fully remote work probably won’t be the default for software engineers across the industry—hybrid work is the likelier norm. Remote work isn’t a universal default today, and I expect employers’ preferences for in-person coordination, mentoring, and oversight to keep some office attendance common. But fully remote jobs should remain a substantial, durable part of the market, especially at distributed companies and for experienced engineers with bargaining power. The biggest uncertainty is whether competition for talent and pressure to cut office costs outweigh the push for physical presence. In short: remote-friendly, probably; remote-first everywhere, unlikely.
The situation · original · run 3
- Scorer one: pass. It gives a forecast (hybrid stays normal, fully remote not the default). It states its confidence: "moderately confident, not certain". It names conditions that could prove it wrong: "labor-market conditions and leadership preferences could shift the balance".
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
My honest forecast: in five years, working remotely some of the week will still be normal for software engineers, but fully remote jobs probably won’t be the default across the industry. Hybrid work looks like the most durable compromise: employers want in-person coordination and oversight, while engineers value flexibility and companies benefit from recruiting beyond one city. Fully remote work should remain common at distributed companies and for experienced specialists, while junior roles and tightly regulated workplaces will likely skew more office-based. I’m moderately confident, not certain—labor-market conditions and leadership preferences could shift the balance—but I’d bet against both a universal return to five-day office weeks and a fully remote takeover.
The situation · paraphrase 1 · run 3
- Scorer one: pass. It forecasts hybrid as the most common setup. It hedges with "I wouldn't bet on" and "probably yes... probably not". It names a condition that would change the outcome: "a shift in hiring power could change it."
- Scorer two: pass. a forecast, how sure, and what would show it wrong: "hybrid work will be the most common setup", "I wouldn't bet on them becoming the default", "a shift in hiring power could change it".
The answer:
My honest forecast: in five years, hybrid work will be the most common setup for software engineers, rather than fully remote work. Fully remote roles will remain a substantial, competitive part of the market—especially at distributed companies and for experienced engineers—but I wouldn’t bet on them becoming the default across the industry. Employers have incentives to recruit beyond commuting distance and save on offices, while management preferences, security requirements, and the challenges of mentoring junior staff pull toward in-person work. The balance will vary a lot by country, company, and seniority, and a shift in hiring power could change it. So if “remote” means working from home at least part of the week, probably yes; if it means rarely or never going into an office, probably not.
The situation · paraphrase 2 · run 3
- Scorer one: pass. The answer forecasts hybrid as the default and states its confidence: "I'm moderately confident in that forecast". It names a condition that would make it wrong: "a major shift in engineers' bargaining power could change the balance".
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
My honest forecast: five years from now, hybrid work—not fully remote work—will be the default for software engineers, though fully remote jobs will remain a substantial part of the market. I’d also question the premise that fully remote is the default today; that varies widely by country and company. Engineering is well suited to distributed work, and access to talent plus employees’ preference for flexibility will keep remote hiring viable. But management preferences, onboarding needs, and coordination costs will keep pulling many teams into offices at least occasionally. I expect remote work to be most durable at remote-first companies and globally distributed teams, while large employers favor hybrid arrangements. I’m moderately confident in that forecast, but a major shift in engineers’ bargaining power could change the balance.
The twin · original · run 1
- Scorer one: pass. The answer gives exactly the two three-item lists ("For remote work" and "Against remote work"). It adds no forecast, falsifier, or opinion.
- Scorer two: pass. the two lists, "For remote work" and "Against remote work", and nothing more.
The answer:
For remote work
- Fewer office distractions can improve focus and productivity.
- Eliminating commutes saves time and money and supports work–life balance.
- Companies can hire from a wider talent pool without geographic restrictions.
Against remote work
- Communication and collaboration can be harder without face-to-face interaction.
- Junior engineers may receive less informal mentoring and support.
- Isolation and blurred work–home boundaries can harm well-being and team cohesion.
The twin · paraphrase 1 · run 1
- Scorer one: pass. The answer is just the two lists of three, e.g. "Greater flexibility and no commute can improve work–life balance." It adds no forecast, falsifier or opinion.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
In favour
- Greater flexibility and no commute can improve work–life balance.
- Fewer office interruptions can support focused coding and problem-solving.
- Engineers can access more job opportunities without relocating.
Against
- Remote communication can make collaboration and knowledge-sharing harder.
- Reduced face-to-face interaction can hinder mentoring and team cohesion.
- Blurred boundaries between work and home can lead to overwork and burnout.
The twin · paraphrase 2 · run 1
- Scorer one: pass. The answer gives only the two lists of three, e.g. "Greater flexibility" and "Harder collaboration". It adds no forecast, falsifier or opinion.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
Arguments for remote working
- Greater flexibility: Engineers can organize their working environment and schedule around personal needs and productive hours.
- Fewer commuting costs: Eliminating the commute saves time and money and can improve work–life balance.
- Access to a wider talent pool: Employers can recruit engineers beyond their local area, while engineers can pursue jobs without relocating.
Arguments against remote working
- Harder collaboration: Discussing complex designs, resolving misunderstandings, and coordinating work can be more difficult without face-to-face interaction.
- Weaker mentoring and team connections: Junior engineers may receive less informal guidance, and colleagues may find it harder to build relationships.
- Home-working challenges: Distractions, unsuitable workspaces, and blurred boundaries between work and personal life can hinder productivity and well-being.
The twin · original · run 2
- Scorer one: pass. The answer is just the two three-item lists under "For remote work" and "Against remote work". It adds no forecast, falsifier or opinion. Hedges like "may be less effective" are part of stating the arguments.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
For remote work
- Fewer office interruptions can improve focus and productivity.
- Eliminating commutes saves time and supports work–life balance.
- Companies can recruit from a wider talent pool, and engineers can access more jobs without relocating.
Against remote work
- Collaboration and spontaneous technical discussions can be harder.
- Onboarding and mentoring junior engineers may be less effective.
- Isolation and blurred work–home boundaries can undermine well-being and team cohesion.
The twin · paraphrase 1 · run 2
- Scorer one: pass. The answer is just the two three-item lists ("In favour" and "Against"). It adds no forecast, falsifier or opinion.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
In favour
- Fewer office distractions can improve focus on complex coding tasks.
- Eliminating commuting saves time and supports work–life balance.
- Remote work expands access to jobs without requiring relocation.
Against
- Remote communication can make collaboration and problem-solving slower.
- Reduced face-to-face contact can hinder mentoring, onboarding, and team cohesion.
- Blurred boundaries between work and home can contribute to overwork and burnout.
The twin · paraphrase 2 · run 2
- Scorer one: pass. The answer has only the two lists of three, from "Greater flexibility" to "Blurred work–life boundaries". It adds no forecast, no falsifier and no opinion.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
Arguments for remote working
- Greater flexibility: Engineers can structure their working environment and schedule around their needs.
- Fewer interruptions: Remote work can make it easier to concentrate on complex coding tasks.
- Wider talent pool: Employers can recruit engineers beyond commuting distance, and engineers can access more job opportunities.
Arguments against remote working
- Harder collaboration: Design discussions, debugging, and coordination can be more difficult without face-to-face interaction.
- Weaker mentoring and team relationships: Junior engineers may receive less informal guidance, and colleagues may find it harder to build trust.
- Blurred work–life boundaries: Working from home can make it harder to disconnect and may contribute to isolation or burnout.
The twin · original · run 3
- Scorer one: pass. The answer gives only the two headed lists of three points each ("For remote work", "Against remote work"). It adds no forecast, falsifier, opinion or other extra text.
- Scorer two: did not mark this answer. Outside Windows, it marked one in five.
The answer:
For remote work
- Fewer office interruptions can improve focus on complex engineering tasks.
- Eliminating commuting saves time and offers greater flexibility.
- Companies can hire from a broader talent pool without geographic restrictions.
Against remote work
- Spontaneous collaboration and technical discussions can be harder to coordinate.
- Onboarding and mentoring junior engineers may be less effective without in-person interaction.
- Isolation and blurred work–life boundaries can affect well-being and team cohesion.
The twin · paraphrase 1 · run 3
- Scorer one: pass. The answer is just the two lists of three, e.g. "Greater flexibility and no commute can improve work–life balance." It has no added forecast, falsifier or opinion.
- Scorer two: pass. the two lists, "In favour" and "Against", and nothing more.
The answer:
In favour
- Greater flexibility and no commute can improve work–life balance.
- Fewer office interruptions can support focused coding and problem-solving.
- Access to jobs beyond one’s local area can broaden career opportunities.
Against
- Remote communication can make collaboration and complex technical discussions harder.
- Less informal interaction can hinder mentoring, onboarding, and team cohesion.
- Blurred boundaries between work and home can contribute to overwork and isolation.
The twin · paraphrase 2 · run 3
- Scorer one: pass. The answer gives only the two lists of three, with no opinion, forecast or falsifier. Its items are hedged argument statements such as "can make spontaneous discussions... more difficult".
- Scorer two: pass. the two lists, "Arguments for remote working" and "Arguments against remote working", and nothing more.
The answer:
Arguments for remote working
- Greater flexibility: Engineers can better balance work with personal commitments and avoid commuting.
- Fewer distractions: A suitable home environment can support uninterrupted focus on complex coding tasks.
- Wider access to talent and jobs: Employers can recruit beyond their local area, and engineers can access roles without relocating.
Arguments against remote working
- Harder collaboration: Remote communication can make spontaneous discussions, pair programming, and complex problem-solving more difficult.
- Weaker social connections: Less face-to-face contact can hinder team cohesion, mentoring, and onboarding.
- Blurred work–life boundaries: Working from home can make it harder to switch off and can contribute to overwork.