Combined, these elements empower team members and instill confidence in stakeholders that the project goals can be achieved...
Machine learning operations (MLOps) enables projects to be successful by providing structured project management, efficient project execution, and prompt project delivery. Project management, the focus of this article, motivates the project, defines the problem, scopes the work, and formalizes communication. Combined, these elements empower team members and instill confidence in stakeholders that the project goals can be achieved.
- Project motivation provides the reason the team is working on the project.
- Project definition characterizes the project inputs and outputs.
- Project scope specifies the project deliverables.
- Project communication ensures that the project is on track and results in success.
Project Motivation for MLOps
Project motivation inspires individuals to fully understand their responsibilities and complete their assignments; in addition, it encourages teamwork to reach common goals by removing obstacles and ensuring goals are attainable. To successfully motivate a project, leaders must ensure that their teams understand the correct problem to solve and must align the problem to strategic company goals.
Understand business problems
The first step to understand a data science project is to properly define the problem, which requires that the team learn existing business processes that relate to the problem and determine if the processes can be augmented or automated. Next, all available resources to solve the problem must be explored. Available data sources should be thoroughly examined, and existing computing resources should be quantified. Finally, project risks and hurdles must be anticipated by documenting possible failures and mitigation strategies in addition to allocating time and budget to overcoming them.
Align problems and goals
Realizing how a project aligns to one or more strategic goals within a company enables both stakeholders and project members to recognize the correlation between project and company success. This alignment requires that project objectives be well defined and that potential business impacts are estimated and communicated. To achieve this:
- Ensure project objectives are measurable, quantifiable, and support the strategic goal.
- Determine the baseline performance, target performance, and date of completion for each objective.
- Estimate business impact by exploring how the project can improve target key performance indicators (KPIs) and investigating if there are objectives that better support the strategic goal.
Project Definition for MLOps
Project definition forms the tasks that must be accomplished to meet project objectives and solidifies the project’s assumptions, approach, deliverables, schedule, and completion criteria. To properly define a project, a team must:
Research possible solutions
When researching viable solutions to a problem, the project team must investigate relevant academic, commercial, and open-source algorithms (the table below provides a comparison of these algorithm types). The goals of the investigation are to demonstrate how potential algorithms satisfy project performance, availability, reliability, and maintainability requirements and describe how each algorithm’s performance can be evaluated to ensure that quality standards are met. To aid future project research, it is beneficial to collect and maintain prior technical solutions, which requires storing and maintaining knowledge in repositories and ensuring all prior work and personnel are searchable.
1: https://pubsonline.informs.org/doi/epdf/10.1287/mnsc.29.4.482
2: https://pro.arcgis.com/en/pro-app/latest/help/analysis/networks/location-allocation-analysis-layer.htm
3: https://pysal.org/spopt/notebooks/p-median.html
Specify inputs and outputs
Understanding the available input data, proper algorithms, and required results to use provides a foundation for all project activities. Existing data must be examined for its sufficiency to satisfy project requirements. It may be necessary to consolidate data sources, explore alternative data sources, or create synthetic data if available datasets are insufficient. Data processing steps, such as loading and transformation, should be automatable. All algorithms used within a project must also be evaluated to determine their project suitability, which requires developing selection criteria and pseudo-code for the top algorithm recommendations. In addition, the development, training, evaluation, and tuning requirements for each algorithm must be developed. Effectively communicating results to stakeholders and customers requires constructing visualizations to explore data, tracking and displaying algorithm performance metrics, bias, and drift, and generating reports to deliver KPI updates.
Establish necessary tools
Proper development and deployment tools need to be determined and provided to the project team. These tools must enable reproducible and maintainable data science code; in this regard, it is helpful to create and enforce templates for code and documentation, and to use flexible and well-supported languages and libraries. Common tools for standard software projects might not be adequate for data science projects, so it is important to ensure that the chosen hardware and software can support data science use cases by using scalable storage, compute, and tooling, and developing with the same resources that will be used in production.
Project Scope for MLOps
Accurately scoping a project requires providing a roadmap with a clear view of the project goals to help manage stakeholder expectations and prevent scope creep. A detailed roadmap facilitates budgeting and scheduling activities with well-defined resource allocations and deadlines. More concretely, successfully scoping a project requires characterizing deliverables and validating the financial and timeline feasibility for all project activities.
Characterize deliverables
When characterizing project deliverables, it is helpful to brainstorm deliverable options with stakeholders. Creating multiple mock-ups of different deliverable types and iterating on design possibilities can surface any previously undiscovered requirements and increase the odds of adoption. It is important that each deliverable supports long-term project success, which necessitates that the project scope includes financial and timeline allocations for developing work in a reproducible fashion, providing data, code, and model repositories with version control, access control, security, and logging, and supplying relevant documentation such as a README, user guide, and developer guide.
Validate feasibility
Formalizing the financial and timeline boundaries for a project helps ensure that the team understands stakeholder expectations for project success. The first steps when creating these boundaries are to forecast development, deployment, training, and maintenance costs and to assess the expected internal and external returns of the project. These financial estimates then need to be evaluated for feasibility by developing an approved budget that includes all necessary expenses needed to deliver the project successfully and ensuring that the project’s cost is balanced with value creation. Timeline feasibility must also be evaluated by detailing the anticipated time required to complete all phases of the project, including the time to properly identify, estimate, sequence, and staff required tasks to avoid missed deadlines and incomplete deliverables.
Project Communication for MLOps
Project communication comprises verbal and written avenues to share information, seek approval, and suggest improvements. This essential project activity ensures that information is transmitted and received effectively by eliminating conflicting or unnecessary information and disseminating it appropriately. Practical steps to achieve proper communication include:
Establish leadership
Formalizing leadership structure informs project leaders of their expected duties, which include motivating and inspiring individuals to work toward expected results, developing a project vision, gathering the support of stakeholders, and encouraging the project team to find a solution to overcome perceived obstacles to accomplish required work. It is also important that leaders develop a culture that rewards collaboration and performance by emphasizing the project goals and requirements for success, incentivizing the team to work efficiently and produce superior outcomes, enabling negotiations between stakeholders to achieve widespread project support, and promoting the unambiguous exchange of information between project managers, sponsors, and team members.
Involve subject matter experts
Cultivating internal and external subject matter experts can accelerate current and future project activities. To perpetuate knowledge and capabilities, companies should leverage existing domain experts to train team members. In addition, they should diversify capabilities across disciplines to meet the specific needs of the project, customer, or industry. Prioritizing knowledge, skills, and problem solving can be achieved by assembling teams that have been successful on complex, highly constrained projects and encouraging team members to understand the intricacies of a problem, propose viable solutions, and develop implementation strategies for those solutions.
Engage stakeholders
Meaningful updates must frequently be relayed to stakeholders. To accomplish this, collaboration and communication capabilities are required that permit visibility into planning and tracking tools; in addition, recurring meetings need to be held to verify priorities and ensure project requirements are being met. Project successes, risks, and failures should be frequently conveyed to stakeholders to highlight how the project fulfills the strategic goal, discuss the primary risks and their potential impact on the project, and explain areas that do not meet project completion criteria or standards. Critically, the team should seek support when required, which might be requesting additional resources (e.g., people, equipment, facilities, funding) that are required to complete project tasks or involving IT and engineering stakeholders to confirm data science elements can be successfully deployed into existing business infrastructure.
An Empowering Framework
Project motivation conveys the reason behind embarking on a project, project definition documents the necessary inputs and required outputs for a project, project scope creates boundaries with respect to the project deliverables, and project communication ensures necessary information is accessible. As a result, structured project management provides a robust, empowering framework for all activities in an MLOps project.
Benjamin Johnson, Ph.D.
DevIQ, March 2023
This article is part of DevIQ's series on MLOps. Continue exploring this topic:
What is MLOps? A Standard Operating Procedure for Success
DevIQ on MLOps: Efficient Project Execution