Metaflow - BKJackson/BKJackson_Wiki GitHub Wiki

Show an app.py file's documentation

python 00-helloworld/helloworld.py show

Run an app

python 00-helloworld/helloworld.py run  

Helloworld example Metaflow code and run output

Note:

• create a class that calls FlowSpec
• each step calls the next step in the flow with self.next()
• start and end steps are required

## Code in helloworld.py  

from metaflow import FlowSpec, step


class HelloFlow(FlowSpec):
    """
    A flow where Metaflow prints 'Hi'.

    Run this flow to validate that Metaflow is installed correctly.

    """
    @step
    def start(self):
        """
        This is the 'start' step. All flows must have a step named 'start' that
        is the first step in the flow.

        """
        print("HelloFlow is starting.")
        self.next(self.hello)

    @step
    def hello(self):
        """
        A step for metaflow to introduce itself.

        """
        print("Metaflow says: Hi!")
        self.next(self.end)

    @step
    def end(self):
        """
        This is the 'end' step. All flows must have an 'end' step, which is the
        last step in the flow.

        """
        print("HelloFlow is all done.")


if __name__ == '__main__':
    HelloFlow()



## Output from $ python 00-helloworld/helloworld.py show:  
(py37) JSFs-MBP:metaflow-tutorials jacquelinefloyd$ python 00-helloworld/helloworld.py show
Metaflow 2.0.5 executing HelloFlow for user:jacquelinefloyd

A flow where Metaflow prints 'Hi'.

Run this flow to validate that Metaflow is installed correctly.

Step start
    This is the 'start' step. All flows must have a step named 'start' that
    is the first step in the flow.
    => hello

Step hello
    A step for metaflow to introduce itself.
    => end

Step end
    This is the 'end' step. All flows must have an 'end' step, which is the
    last step in the flow.


## Output from $ python 00-helloworld/helloworld.py run:  

Metaflow 2.0.5 executing HelloFlow for user:jacquelinefloyd
Validating your flow...
    The graph looks good!
Running pylint...
    Pylint is happy!
2020-07-20 16:11:30.457 Workflow starting (run-id 1595279490449536):
2020-07-20 16:11:30.462 [1595279490449536/start/1 (pid 52152)] Task is starting.
2020-07-20 16:11:30.922 [1595279490449536/start/1 (pid 52152)] HelloFlow is starting.
2020-07-20 16:11:30.964 [1595279490449536/start/1 (pid 52152)] Task finished successfully.
2020-07-20 16:11:30.970 [1595279490449536/hello/2 (pid 52158)] Task is starting.
2020-07-20 16:11:31.424 [1595279490449536/hello/2 (pid 52158)] Metaflow says: Hi!
2020-07-20 16:11:31.463 [1595279490449536/hello/2 (pid 52158)] Task finished successfully.
2020-07-20 16:11:31.468 [1595279490449536/end/3 (pid 52163)] Task is starting.
2020-07-20 16:11:31.929 [1595279490449536/end/3 (pid 52163)] HelloFlow is all done.
2020-07-20 16:11:31.972 [1595279490449536/end/3 (pid 52163)] Task finished successfully.
2020-07-20 16:11:31.973 Done!  

Read in a CSV file with IncludeFile function

movie_data = IncludeFile("movie_data",
                         help="The path to a movie metadata file.",
                         default=script_path('movies.csv'))

Function to read in command line parameters at run time

genre = Parameter('genre',
                 help="Filter movies for a particular genre.",
                 default='Sci-Fi')