AI Safety & Alignment
Reducing harm from models, products, agents, and the environments they can act in
What is AI safety?
AI safety is the work of making AI systems behave reliably enough for their intended use while reducing foreseeable harm.
That includes more than filtering bad answers. Safety can involve:
- Model behavior — what the model tends to produce
- System design — what tools, data, memory, and permissions surround the model
- Deployment controls — who can use the system, for what, and under what limits
- Monitoring and response — detecting failures and recovering when something goes wrong
- Governance — laws, standards, organizational processes, and accountability
Alignment is one part of safety
Alignment asks whether a system's behavior matches the intentions, constraints, and values humans actually want it to follow.
A system can fail because the goal was specified badly, because the model behaves unexpectedly, because the environment gives it too much authority, or because operators fail to detect a problem. Real-world safety therefore cannot be reduced to one "alignment problem."
Risk depends on capability and access
The same model can present very different risks depending on what it can access.
A text-only assistant with no external permissions is different from an agent that can:
- Browse the web
- Execute code
- Access private files
- Use credentials
- Send messages
- Purchase goods
- Change production systems
For agentic systems, least privilege, sandboxing, network controls, approval gates, monitoring, rate limits, and rollback can matter as much as output filtering.
What you can do
- Verify important AI outputs against reliable sources.
- Do not give an AI system secrets or permissions it does not need.
- Use stronger human approval for higher-impact actions.
- Treat model confidence and fluent language as different from evidence.
- Report unsafe behavior and preserve enough logs to investigate incidents.
References
Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking Press.
Hendrycks, D., Mazeika, M., & Woodside, T. (2021). An Overview of Catastrophic AI Risks. arXiv preprint.
Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press.
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016). Concrete Problems in AI Safety. arXiv preprint.
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., et al. (2022). Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv preprint.
Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., & Amodei, D. (2017). Deep Reinforcement Learning from Human Preferences. Advances in Neural Information Processing Systems (NeurIPS).
Citation Note: Some citations are open access (arXiv or DOI, where linked). Others are books or journal articles that may sit behind a publisher paywall. Use the linked DOI or arXiv when available. If you notice any citation errors, please let us know.