Plugins can create their own UI elements, and even spawn their own OpenGL windows.

The following general steps are required if you want to make your own plugin and use it within Pupil:

• Fork the pupil repository (if you haven't done this already) and create a branch for your plugin. Try to make commits granular so that it can be merged easily with the official branch if so desired.
• Create a new file
  • In /capture if your plugin only interacts with Pupil Capture's World process.
  • In /player if your plugin only interacts with Pupil Player.
  • In /shared_modules if your plugin is used in both Pupil Capture and Pupil Player
• Inherit from the Plugin class template. You can find the base class along with docs in plugin.py. A good example to base your plugin on is display_recent_gaze.py
• Write your plugin
• Use your plugin
  • Pupil Player
    • Import your plugin
    • Add your plugin to the user_launchable_plugins list in player/main.py.
  • Pupil Capture - World Process
    • Import your plugin
    • Add your plugin to the user_launchable_plugins list in `capture/world.py'
• Select plugin from the "Open plugin" in the main window that launches to use your plugin.

Another way to start plugin development, is to use an existing plugin as a template. For example, you could copy the vis_circle.py plugin as a starting point.

renaming it to, for example, open_cv_threshold.py.

Now you could give a new name to the class name:

class Open_Cv_Threshold(Plugin):

Rename its super reference:

super(Open_Cv_Threshold, self).__init__(g_pool)

Describe what your new plugin will do for yourself in the future and for future generations:

class Open_Cv_Threshold(Plugin):
"""
  Apply cv2.threshold filter to the world image.
"""

Rename its reference in the persistence method:

 def clone(self):
return Open_Cv_Threshold(**self.get_init_dict())

It is good to rename its menu caption as well:

self.menu = ui.Scrolling_Menu('Threshold')

Lets determine its execution order in relation to the other plugins:

self.order = .8

The current execution order is the following:

(describe why some plugins does not require an order, and why ordering is important) (describe if a plugin have an inherited order)

You can allow (not_unique) or not (unique) multiple instances of this same plugin:

self.uniqueness = "unique"

(present the string variables that can be used in the uniqueness attribute, maybe there is inconsistency right now)

(describe how to safely remove unneeded parameters)

Finally, lets implement what our new Plugin will do. Here we choose to apply an OpenCv threshold to the world image and give us proper feedback of the results, in real time. Good for OpenCv and related studies. It is possible by the update method:

(describe world frame structure; maybe linking to trusted OpenCv docs)

def update(self,frame,events):
   img = frame.img
   height = img.shape[0] 
   width = img.shape[1] 
   
   blur = cv2.GaussianBlur(img,(5,5),0)

   edges = []
   threshold = 177
   blue, green, red = 0, 1, 2

   # apply the threshold to each channel 
   for channel in (blur[:,:,blue], blur[:,:,green], blur[:,:,red]):
      retval, edg = cv2.threshold(channel, threshold, 255, cv2.THRESH_TOZERO)
      edges.append(edg)
   
   # lets merge the channels again
   edges.append(np.zeros((height, width, 1), np.uint8))
   edges_edt = cv2.max(edges[blue], edges[green])
   edges_edt = cv2.max(edges_edt, edges[red])
   merge = [edges_edt, edges_edt, edges_edt]
   
   # lets check the result
   frame.img = cv2.merge(merge)
    

(describe events structure)

(describe PyGlui menu integration, for example, with a slider to the threshold value and illustrate how achieve persistence of the parameter)

from open_cv_threshold import Open_Cv_Threshold 

...

user_launchable_plugins = ..., Open_Cv_Threshold 

To export a video with the Custom Plugin visualization all you need to do is...

(describe export integration)

If you want to close the plugin all you need to do is:

self.alive = False # from within your Plugin
# or
your_plugin.alive = False # from outside

This will first call Open_Cv_Threshold.cleanup() and then destroy the class instance.