Terms & Definitions

Intelligent forecast

Socratic's intelligent forecast uses machine learning—specifically, Monte Carlo simulation—to predict completion dates for any body of work.

How Monte Carlo simulation works

Monte Carlo is a statistical technique used to model complex systems across a wide range of fields—traffic control, stock predictions, and nuclear reactors, to name a few. The idea is to use the law of large numbers to predict how a complex system might behave. Monte Carlo runs scenarios using our best empirical data and knowledge of the "rules of physics" for the system at hand. By collecting results from enough randomized scenarios, we can get an idea not only of what is most likely to occur, but also visualize the range of possible outcomes.

The basic steps and inputs are as follows:

1. Set up N issues with random (but representative) durations.
2. Set up a pool of M people to complete the issues.
3. Assign the N issues to the M people.
4. Determine the completion dates for each issue.
5. Construct a burndown chart from these dates.
6. Repeat steps 1-5 Q times.

In creating a body of work, you provide the number of issues (N) and the people that will work those issues (M); Socratic does the rest. This includes randomly-sampled issue durations based on your organization's historical actuals, which is needed for the first step. In other words, the data is entirely personalized to your teams.

We've set the number of model runs (Q) to 1,000.. The more you do, the more statistical significance you gain, but it also starts to absorb some serious compute cycles. In our own analysis, a thousand runs strikes a nice balance between system responsiveness and forecast reliability. Simply put, this model reveals all of the possible durations if the same body of work was delivered by the same people, a thousand times.

With Monte Carlo simulation, we’re able to absorb a number of complexities that impact delivery and that are otherwise invisible to the naked eye. These include nuances like:

• The rate at which issues fall idle;
• The number and relationship of blocked issues;
• The amount of rework involved before an issue is completed;
• The amount of inevitable scope creep in a project;
• The impact of holidays on people’s availability;
• The participation of people with specialized skills.

For active bodies of work, we produce a timeline graph that shows two things:

1. A completed issues line, rising to meet a scope line: that is, a burn-up;
2. A forecast for time to finish, based on your historical actuals. (Monte Carlo actually delivers a likely date range, with outliers on the short and long side of the forecast, and a midpoint date. The midpoint date is what we use for display.)

FAQs

Does Socratic expect inputs on issue complexity?

No. Our model is concerned only with historical actuals, not manual guesstimates like story points. The inevitable differences in issue size or complexity is solved through the law of large numbers. This also means that no special effort is required to make your issues approximately the "same size." You simply break down your work as best you can, and leave the rest to us!

How do you account for unassigned work?

When a body of work has unassigned issues, we use an "average contributor" for the model—meaning, we look at your organization's historical average time to complete. As issues are assigned, our model then personalizes to the assignee(s).

Note that in cases where some issues are assigned and others are unassigned, our model assumes that the known assignees will also be the people to deliver the unassigned work. This means the model determines when the assigned issues will be completed by person, and assigns the unassigned issues based on when each person will have capacity.

As an example: say an epic has ten child issues. Only one of the ten issues is currently assigned. The model will build a forecast by assuming the known assignee must first complete their current assigned issue, and then pick up the remaining nine issues and work them sequentially. As issues are assigned out to additional people, the model adjusts automatically, using the number of known assignees as the available pool of people to deliver the remaining unassigned issues.

Cycle time

For cycle time, Socratic automatically measures the elapsed calendar days from the time work starts on an issue, to the time it's completed. (Work starts when an issue is moved from a backlog status to any active status.) So, if work begins on a Monday and finishes the following Monday, its cycle time is 7 days.

Efficiency

Efficiency reflects how much work time is spent in a normal, flowing state—that is, issues moving left to right through your Jira statuses—versus those that have fallen into an exception states. Where possible, Socratic infers these exception states automatically:

Rework

Time spent on issues that went backward in the workflow, either from a later active status (e.g. "Testing") to a prior one (e.g. "In Development"), or from the Done status back to an active status.

Deprioritized

Time spent on issues that move from an active (In Progress) status back to a waiting (To Do) status.

Blocked

Time issues spend as blocked. This is the one state we can't infer by ticket movement. When you mark a ticket as blocked in Jira, we track the amount of time it spent in that blocked state.

Bear in mind, efficiency is a measure of time worked by state, not ticket counts. For example, if an issue takes 5 days to move from start to finish, but along the way is blocked for one day, its flow efficiency is 80 percent. Four of its five days (80%) were in a normal, flowing state, with one day (20%) in a blocked state.

Burning worked time in exception states—blocked, deprioritized, rework—is a natural part of software engineering. The target of efficiency is not and should not be 100 percent. The aim is simply to understand how much time is spent in any of these exception states, and to mitigate it whenever it appears high.

Momentum

The aim of momentum is to show you at a glance what work is moving versus stalling or regressing.

To depict momentum, we use a month-over-month variance (delta) chart. Each thin vertical bar shows the change in work completed month over month; the length encodes the size of the change.

• Blue: momentum increase. Meaning, more work was completed than was added in the month.
• Orange: momentum decrease. More work was added than was completed in the month.

Resource contention

Resource contention depicts the percentage of issues assigned to the team that fall outside the body of work in question.

Put simply, low percentages mean low resource contention. In this case, low is better.

Let's say a given body of work, such as an epic, has issues assigned across ten people. If all ten of those people are working only issues related to this epic, the resource contention is 0 percent. The team has no issues assigned that fall outside this epic.

If the resource contention shows e.g. 64 percent, this signals a high degree of resource contention. Only a minority of the team's assigned issues (36 percent) relate to the epic in question. Most of their assigned issues are for other work.