example: mapreduce - modrpc/info GitHub Wiki

Table of Contents

Example #1: MapReduce

By building a simple MapReduce framework with ModPy, we can illustrate how ModPy resources can be used to build a distributed application. In this framework, we will slightly twist the Reduce phase so that reduction will happen through paralel prefix computation.

Nodes

We will create two types of nodes: master node and worker nodes and data node. For simplicity, assume there is one master node and 10 worker nodes. The master node coordinates the overall workflow, worker nodes perform Map and Reduce functions, and the data node stores the data to operate on.

Master Node

def map_reduce(nworkers):
  # initialization phase
  # map phase
  for i in range(nworkers):
    await modpy.
  # reduce phase
  # print result

Example #2: Network Flows

Nodes

There are three types of nodes: start node, finish node, internal nodes.

Example #3: Raft

Example #4: Life Game

Example #5: File Transfer

A digital system is typically described by finite state machines which is synchronized to a global clock.

Example #6: MIPS Processor

NODE0: Clock Generator

Let's create a node which generate clock signals.

@modpy.event
def clock_tick_event()
  return True

@modpy.timer
def clock_timer()
  return yield from asyncio.sleep(1)

@modpy.proc
def clock_generator():
  while True:
    yield from modpy.wait_timer("clock_timer")
    yield from modpy.publish("clock_tick_event", True)

NODE1: Counter

counter = 0

@modpy.proc
def counter():
  global counter
  while True:
    yield from modpy.wait_for("node0", "clock_tick_event")
    counter = counter + 1
⚠️ **GitHub.com Fallback** ⚠️