The DFP example workflows in Morpheus are designed to scale up to company wide workloads and handle several different log types which resulted in a large number of moving parts to handle the various services and configuration options. To simplify things, the DFP workflow is provided as two separate examples: a simple, "starter" pipeline for new users and a complex, "production" pipeline for full scale deployments. While these two examples both perform the same general tasks, they do so in very different ways. The following is a breakdown of the differences between the two examples.
This example is designed to simplify the number of stages and components and provided a fully contained workflow in a single pipeline.
Key Differences:
- A single pipeline which performs both training and inference
- Requires no external services
- Can be run from the Morpheus CLI
This example is designed to illustrate a full-scale, production-ready, DFP deployment in Morpheus. It contains all of the necessary components (such as a model store), to allow multiple Morpheus pipelines to communicate at a scale that can handle the workload of an entire company.
Key Differences:
- Multiple pipelines are specialized to perform either training or inference
- Requires setting up a model store to allow the training and inference pipelines to communicate
- Organized into a
docker composedeployment for easy startup - Contains a Jupyter notebook service to ease development and debugging
- Can be deployed to Kubernetes using provided Helm charts
- Uses many customized stages to maximize performance.
Guides for each of the two examples can be found in their respective directories: The Starter Example and The Production Example