Formal Methods: Just Good Engineering Practice? - Marc's Blog

🌈 Abstract

The article discusses the importance of using formal methods in software engineering, especially for large-scale, distributed, and critical low-level systems. It argues that formal methods can significantly increase the speed and efficiency of the software building process by reducing rework and the cost of change.

🙋 Q&A

[01] The importance of formal methods in software engineering

1. Questions related to the content of the section?

• The author believes that formal methods are an important part of good software engineering practice, especially for those working on large-scale systems, distributed systems, or critical low-level systems.
• The author argues that if you're a software engineer and not using formal methods as part of your approach, you're probably wasting time and money.
• The author explains that formal methods can save on the cost of rework and get interface changes out of the way earlier, which can significantly increase the speed and efficiency of the software building process.

2. Why does the author believe formal methods are important for software engineering?

• The author believes formal methods are important because:
  • They can reduce the cost of rework, which is a significant issue in software engineering due to the ability to start construction before design is complete.
  • They can get interface changes out of the way earlier, which reduces the cost of change later on when the system has customers.
  • They can help explore possible optimizations and find the constraints that really matter, removing the hard trade-off between correctness and performance.

[02] When are formal methods more valuable?

1. Questions related to the content of the section?

• The author suggests that formal methods are more valuable for systems with well-understood requirements, such as large-scale, distributed, and low-level systems.
• For software with heavily evolving or hard-to-formalize user requirements, such as UIs, web sites, and pricing logic implementations, the value of upfront design may be diluted or its costs increased.
• The author suggests that the closer the requirements process is to the laws of physics, the more valuable design and formal design are in the engineering process. The closer the requirements are to users' opinions, the less valuable they are.

2. What factors determine the value of formal methods in software engineering?

• The value of formal methods depends on the nature of the software requirements:
  • For software with well-understood, static requirements (e.g., large-scale, distributed, low-level systems), formal methods are more valuable as they can reduce rework and the cost of change.
  • For software with heavily evolving or hard-to-formalize user requirements (e.g., UIs, web sites, pricing logic), the value of upfront design may be diluted or its costs increased.
  • The closer the requirements process is to the laws of physics, the more valuable formal design is. The closer the requirements are to users' opinions, the less valuable formal methods are.

[03] Specific tools and approaches for formal methods

1. Questions related to the content of the section?

• The author mentions specific tools they have found useful in their domain of big cloud systems, such as TLA+ and P.
• The author also references a survey of methods in the paper "Using Lightweight Formal Methods to Validate a Key-Value Storage Node in Amazon S3" as a good place to start.
• The author emphasizes that the goal is not just to verify implementations, but to use tools like TLA+ and P to think through designs more quickly and concretely before building.
• The author shares an example where using TLA+ helped a team verify an "aggressive optimization", showing how formal methods can help build faster systems, not just correct ones.

2. What are some of the specific tools and approaches the author recommends for using formal methods in software engineering?

• The author mentions the following tools as useful in their domain of big cloud systems:
  • TLA+
  • P
• The author also references a survey of methods in the paper "Using Lightweight Formal Methods to Validate a Key-Value Storage Node in Amazon S3" as a good starting point.
• The author emphasizes that the goal is not just to verify implementations, but to use tools like TLA+ and P to think through designs more quickly and concretely before building.
• The author provides an example where using TLA+ helped a team verify an "aggressive optimization", showing how formal methods can help build faster systems, not just correct ones.