As an agency, our business model revolves around time. Our client activities rely on a dedicated number of hours per week worked on a project, and our internal activities follow the same pattern. As such, time tracking is a vital part of our work. Ensuring time is tracked correctly, and time entries meet a minimum quality standard, allows us to be more data-driven in our decisions, provide detailed invoices to our clients and better manage our own projects and initiatives.

Despite being a core activity, we had been having several issues with it not being completed or not being completed properly. A report we ran at the end of 2022 showed our time tracking issues were actually quite severe. We lost approximately one million dollars in 2022 due to time tracking issues that led to decisions made on poor data. It was imperative that we solved the problem.

To help with this issue, we created an evolution of our Pecas opens a new window project. We turned Pecas into a machine learning powered application capable of alerting users of issues in their time entries. In this article, we’ll talk though the business case behind it and expected benefits to our company.

The Time Tracking Problem

Our time tracking issues pre-dated the 2022 end of year report. By that point, we had been having problems for a couple of years, it just wasn’t a big priority. As the company grew, however, the issues multiplied, and got to a point where we needed to prioritize solving the problem.

A detailed analysis of our time tracking data revealed several different issues, both issues that were typically caught by internal processes relying on this information, such as invoicing, and issues that typically remained hidden:

• Billable entries logged to non-billable projects;
• Time that “disappeared” (no logged to a client or internal project) in a given week;
• Potentially billable time that went to internal, non-billable projects instead (time that had been “sold” but wasn’t used as such);
• Vague, non-descriptive entries that harm internal decision making and generate back and forth on invoicing;
• Incorrect entries making it seem like specific activities took much longer (or much less time) than they actually did;
• Delays in our invoicing process caused by entries needing to be fixed;
• More time invested in manual reviews and correcting entries that should be invested elsewhere.

These were some of the main issues we were facing, and as a small company, their impact was even more significant to our projects and our operation overall. We knew it was a problem, and we attempted a few different solutions, including implementing policies around time tracking. They ended up having serious flaws that caused us to reconsider and eventually retract them. But we still had a problem to solve.

At the end of 2022, when we looked at our number for the year, we decided to dive deeper into this data. And the cost of the issues mentioned above became very clear: we lost $1,000,000 dollars due to these issues and their consequences. What this meant is that we had a million dollar problem to solve.

Time tracking issues (timing and quality of entries) are one aspect of a complex problem. Improving time tracking quality was one of the problems we had to solve, and one of significant impact. There were, however, multiple root causes that led to the loss we identified (process problems, service management, communication). Those are being addressed separately and are beyond the scope of this article.

The Solution to Time Tracking Quality

Our main issue was that specific time tracking policies we had implemented didn’t account for nuance. Although delays in time entries being entered into the system and entries logged to incorrect projects decreased, addressing some of the most costly problems we had, honest mistakes were treated in the same way as more serious issues, and the policy was found to be unfair in some cases.

This went against our core values opens a new window and led us to look for a different solution. The main issue was that there was no way to be alerted of honest mistakes in entries before the information was needed, someone reviewed and found the issue manually, or we ran another comprehensive report.

Manual processes for these kinds of tasks are not great. They are expensive and take people away from other activities. We wanted an automated way to monitor and flag entries. We knew from the beginning there was always going to be a human component to it, but if we could reduce the time we spent every week running reports and reviewing and fixing entries, that was already a win.

That’s when we decided to build an internal tool to help with this. Our goal was to reduce the time invested on time tracking by our operations team by automating the bulk of the work to find these issues, and leaving human reviews to a much smaller set of of entries.

This solution would need to be able to:

• Evaluate each time entry;
• Decide whether it is valid or invalid;
• Notify the user of issues found.

The complexity lied in the fact that we’re dealing with text data in free speech form (entry description) combined with several other properties (project, labels, date, billable or non-billable, duration). Accounting for all possible scenarios and issues with hard rules would not work. That’s where machine learning comes into play.

We split the entry classification part of the solution in two:

• Hard-rules checking common entry types for commonly seen issues;
• Binary classification model for the more complex ones.

No solution is perfect, and we knew there were going to be issues that still slipped through the cracks, as well as a need for human review. Our goal was to minimize both.

That’s how the Pecas project was born.

The Business Case

When we decided to build the solution, we were spending between 3 and 5 hours every week on time tracking reports. That meant spending between $30,000 and $50,000 every year just on these reports. As the company grew and we had more people joining the team, the time spent on this was also going to increase significantly.

In summary, we had one million dollars in losses in 2022 alone, and were looking at a current cost of $30,000 to $50,000 per year to run the process manually, increasing every time our team grew. We had a pretty solid case to invest in a solution.

Additional factors that contributed to our decision to go ahead with the project were:

• Good opportunity for experimentation: Pecas is our first tool built with machine learning integration;
• Learning opportunities:
  • It provided a good learning opportunity for team members looking to work with machine learning;
  • It gave our more junior engineers an opportunity to work on building a product from the ground up;
  • It gave interested team members an opportunity to work with different technologies, namely Python and Django.
• Knowledge share: it gave our senior team members an opportunity to knowledge share and mentor more junior team members.

In order to properly evaluate whether building a solution was the right move, we also had to consider implementation and maintenance costs. We had the expertise needed within our team, so we didn’t really need to bring in external help to accomplish what we needed; and even with the added complexity of a machine learning model, we were looking at a small application. To put it into perspective:

Assuming we did nothing, we would continue to incur significant losses year after year. From our data analysis and root cause evaluation, we believe the solution could help reduce the loss by approximately 60%, saving us $600,000. Similarly, the solution can reduce the time spent reviewing time entry reports by 80%, meaning our costs would reduce to $6,000 to $10,000 per year, saving us between $24,000 and $40,000 every year, not accounting for potential growth.

Building the solution would cost approximately 50% of the total we expected to save, and maintaining it, once built, certainly wouldn’t cost as much as we were losing. Pretty good case to build it!

Add to that the knowledge and learning gains, and preserving our culture of team first, and the decision was easy.

Current State

The Pecas app’s first version went live in March and, at that time, supported only filter classification with hard rules. That allowed us to measure user interaction with it and see how (or if) things would improve. It also got us thinking about new ways to leverage the app.

A version of the app with machine learning integrated went live in August, and we have been monitoring it and collecting data. The number of common issues in entries identified has decreased significantly, and timeliness of the logging has greatly improved.

We have found additional use cases for the bot, and created additional alerts for project teams, project managers and our overall Operations team. This has allowed us to identify issues faster and react to them immediately, saving us time, money, and headache in the long run.

We’re still monitoring data and working through results, a preliminary analysis shows that the number of billable time entries logged to non-billable projects in Q3 2023 was 95% smaller than for the same period in 2022, so we’re calling this an win for now as we continue to expand the machine learning and other functionalities.

Conclusion

Machine learning isn’t a magic bullet to all of our problems. In fact, in many cases, it isn’t quite the right solution, and you can go very far with hard rules. There are situations, however, where it is the ideal solution. In those cases, it is a powerful tool to solve very complex problems.

As previously mentioned, an automated tool to aid time tracking quality wasn’t the only solution to this problem. Changes in process were also required, and each case was examined, separately and in conjunction with others, and addressed. But it was a core piece in the strategy, and the results are positive and quite promising.

We specialize in solving complex problems for companies looking to build interesting tools that provide meaningful results. We take a holistic look at the problem, advise on all aspects of the problem, and can help you improve your processes and build the right tool for the right problem.

Got some difficult problems you’d like to solve with software but not quite sure where to start? Unsure if machine learning is the right solution to your problem? Send us a message opens a new window .