Why use Taskmatics Scheduler
Why use Taskmatics Scheduler?
Taskmatics is preparing for its first major release of its flagship application, Taskmatics Scheduler. If you’re a .Net developer you should be excited about this application. To understand why, it helps to understand the motivation behind creating the system in the first place. We believe that the same reasons that compelled us to create Taskmatics Scheduler will be the same set of criteria that drive .Net developers to adopting it.
The Problem
As .Net developers, most of us have been involved with the development of Enterprise class systems. These are generally large, often complex applications that:
- Have encapsulated business logic in code – via services, assemblies, etc… e.g., the ‘Customer’ object that encapsulates all rules for managing a customer
- Encapsulates complex business processes involving multiple objects, e.g. onboarding/ingesting product, creating an order, etc…
- Uses one, or multiple data stores for persistence
- Have one or multiple UI layers, e.g. an administrative app, consumer facing app, etc…
All of these components work together to form our respective ‘systems’; However, problems begin to arise when we need to support batch or offline operations utilizing the business rules already built into these systems. For example:
- At regular intervals we need to ingest new product into our catalog
- At specified times we need to check with a 3rd party to see if there are new orders to add to our system
- ETL activities (import/export data)
- Rebuild indices, aggregate data
Thus, the need for a centralized job management solution in this type of environment is crucial.
A Pseudo Solution
Like many .Net developers, we turned to Windows Task Scheduler for a solution to our job management needs. Windows Task Scheduler provides an ability to start batch or .exe files, has a multitude of scheduling options, and is available on any flavor of Windows servers’ installations.
Issues with approach
Windows Task Scheduler views each task under its management as an independent entity. This approach has some benefits, but also some severe drawbacks that quickly become management headaches as the underlying job infrastructure evolves. Notably:
- No common framework, job infrastructure quickly becomes the wild west
- No centralized logging solution
- No remote management
- Very difficult/clumsy mechanism for using shared files (assemblies)
- No extensibility
- Single server solution
- Laborious to update/maintain existing jobs
(Note: We evaluated SQL Server, Quartz, and ActiveBatch as well. Please stay tuned for pending discussions regarding these products shortcomings and why we were compelled to create Taskmatics Scheduler)
Finally, a Solution!
We concluded that there was not a suitable task management system available that addressed the needs of an enterprise .Net developer, so we decided to write our own. From our experience, we were certain that a task management solution needed to have at least the following:
- Job isolation (a poor performing job can NOT bring down the entire job infrastructure)
- Remote management
- Extensibility
- Common Framework
- Common Logging
- Ability to update jobs while jobs are running
- Ability to leverage common code (assemblies)
- Reporting
- Resource utilization by Job
- Security (access and authorization)
- Ability to scale out to multiple job servers
- High Availability configuration
We knew that there weren’t any applications available in the marketplace that supported our desired feature set, so we decided that the only way to get what we needed was to create our own. So, we did just that.
Our initial version of the application was created for our own internal use. It was a bare bones application that lacked an administrative console and had a very crude configuration system. However, our internal adoption and reliance on that crude system convinced us of the needs for this feature set, so after 2 years of evolution and internal use we re-developed the application with a goal of releasing it to the public.
And thus, Taskmatics Scheduler was born! Taskmatics Scheduler represents all of the knowledge learned from our experience with the initial system and, most importantly, addresses the areas that we found lacking (configuration, installation). The result is a full featured task management system that is a must have for any .Net developer.