As a senior software engineer, I’ve realized that the job entails more than just writing code; in fact, it deviates quite significantly.
Firstly, I dedicate a substantial portion of my work hours to non-technical aspects such as scoping a project, prioritizing tasks, fostering effective communication, etc. Secondly, even when I start implementing a software functionality, the primary challenge lies in meticulously planning the interdependencies among the components involved. Once I complete this planning phase, translating the work into code becomes straightforward.
Based on these observations, I have conceptualized the role of a software engineer as that of a “dependency planner.” In this article, I will delve into the concept of Dependency Planning, explore its importance and describe how I do it.
What is Dependency Planning?
Dependency planning is the process of identifying and documenting the interdependencies among components within the problem scope. A “component” can be a software library, a service, or data. The result of dependency planning is typically a graph that illustrates the dependencies and interfaces between components (however, it’s often presented as a design doc). It’s usually sufficient for the engineers to start implementing the functionality by following the dependency plan.
For example, when building a feature to display membership status on a website homepage, dependency planning would include identifying the necessary elements for successful execution. The final plan should include details such as:
The feature relies on the membership status data from database X.
A controller transfers data from database X to frontend Y.
A renderer in frontend Y renders the UI for the user.
It’s essential to have a clear understanding of the intended deliverables before starting on dependency planning. It’s acceptable to have some details missing as long as they don’t significantly impact the dependencies. For example, uncertainty about the specific page to display the membership status on should hold the planning process, but uncertainty about the membership status options shouldn’t (as it should be easy to extend overtime).
Why Do We Need to Plan Dependencies?
The initial objective is to establish all necessary dependencies and deliver the desired functionality. However, this only meets the minimum requirements.
In practical scenarios, multiple options exist for interdependencies among components to deliver the target functionality. The challenge lies in selecting the best option that aligns with the prioritized system attributes (such as latency, usability, and others). For instance, you may have two paths to retrieve membership status from a database: directly from the database or from cached data on the device. The former provides more accurate data if the status changes frequently, while the latter offers faster data retrieval if low latency is critical. The best approach depends on which attributes that your project prioritizes.
Developing software with the desired attributes requires a comprehensive understanding, which is beyond the scope of this article. However, it’s critical to remember that the essence of dependency planning is not only to get it work — but to get it right.
How to Plan Dependencies?
The desired result of a dependency planning is a dependency graph, which is usually in the developer’s mind and documented in the design doc. This serves as a mental model while executing the below steps. It’s worth noting that when the problem is complex, it’s common to iterate through these steps. As we gather more information, we may uncover additional options or make different decisions. This process continues until we implement a solution and deliver it.
Step 1 — Backtrace Dependencies to Get Options
When building a new functionality, I begin by listing the necessary components it depends on. I create an exhaustive list while disregarding details irrelevant to the dependency graph, such as the specific value type of membership data. For each dependency identified, I repeat the process to determine its dependencies. Through recursion, I identify all the required components.
During the dependency graph creation process, you may encounter multiple paths to establish a dependency. In such cases, list all the options that seem viable and move to the next step.
Step 2 — Make Trade-Offs
While facing multiple dependency graphs to choose from, consider the follow steps as a guide:
Investigate Constraints: Identify constraints that may eliminate certain options. These constraints could be technical limitations, resource availability, or external factors.
List Pros and Cons: For each option, create a list of its advantages (pros) and disadvantages (cons). This will help you understand the strengths and weaknesses of each choice.
Engage with Stakeholders: Collaborate with all relevant stakeholders to compare the pros and cons of each option. Engage in open discussions to gather diverse perspectives and insights.
Decision-Making: Based on the input from stakeholders and your analysis, make a decision on the path to take. Document the decision and the reasons behind it. This documentation serves as a valuable reference for future reference and accountability.
When making trade-offs, I follow two rules of thumb:
“Low Coupling, High Cohesion”: Aim to minimize connections between components and maximize cohesion within each component. Prioritize options that strongly adhere to this rule, as they often result in desirable system attributes.
Minimal Changes: Whenever possible, opt for code changes that are as minimal as feasible. This approach allows for faster progress and often reflects “low coupling, high cohesion.” Keep in mind that the complexity of your problem influences the amount of changes required. Be intentional when adopting designs that require significant changes.
Step 3 — Plan Interfaces
After deciding on a dependency graph, it’s beneficial to examine the interfaces provided by each component. These interfaces commonly take the form of HTTP entries, RPC calls, or library calls. During this step, I will identify the changes planned for these interfaces. This may include adding new interfaces, modifying existing ones, or leaving them unchanged. I will then share the plan with stakeholders for early review to avoid surprises when I send out the actual changes.
Summary — It’s Hard
Dependency planning, like art, lacks a universal formula, and individuals cultivate their own unique styles. Skilled engineers craft dependency plans that are a pleasure to review, while learners continually refine their techniques as they progress in their careers. The difficulty of dependency planning is undeniable, and I’ll conclude this article by exploring the reasons behind this difficulty:
Identifying the dependency is hard: In small software projects with limited users, engineers can easily identify dependencies to deliver new features. However, larger systems with complex tech debts make dependency identification incredibly challenging. Clarifying dependencies requires experience and a clear understanding of desired outcomes.
Making trade-offs requires non-technical knowledge: Trade-offs in design decisions vary significantly depending on the project’s context. Features intended for millions of users in the near future have different requirements than prototypes that may ultimately be discarded. Making informed trade-offs demands a comprehensive understanding of project goals, including timelines, anticipated outcomes, and stakeholder interests.
Best solution usually requires experience: Often, we can identify a solution that requires minimal changes and intuitively “feels right” as a resolution to our problem. Such a solution typically comes from a senior engineer who understands the problem and the system deeply. They can draw upon their experience and knowledge to quickly assess the situation and come up with an elegant and effective solution.