Inheritance diagram for phSimpleVision:
The capture class allows the setting of the capture device and the channel setting as well as all the picture settings except the capture format as it is rarely changed for any purposes in normal vision applications. Any conversions can be performed after the image is captured by any of the filters if necessary.
The pipeline can be altered to run different filters through the use of the member functions which have been abstracted to perform certain vision functions (train_color) where necessary.
The displays being used include the NetDisplay (using JPEG output) and a normal X11Display class. The NetDisplay class is intended to be used over lower bandwidth links (such as wireless) to prevent complete saturation using raw data when several people are using the same WAP. There are two of each display for displaying the raw capture output and the pipeline processed output.
The phSimpleVision class encapsulates a very simple vision framework using Phission components. The driving purpose behind this class is to handle many of the mundane details of setting up a vision system in Phission. This allows the phSimpleVision class to require little or no prior vision processing or Phission experience to allow a beginner student to get started with vision processing in as little time as possible.
Phission was initially built to provide the best performance possible for the given hardware on which it is being run. The design of an entire system consists of sources (capturing), sinks (displays) and sink-source (pipelines) objects. Setting up a vision system requires a good deal of setup code which not all users are interested in knowing right away. There is also a good deal of error checking and memory/class allocations to perform, too. The phSimpleVision class performs all these tasks to set up the more common example of a simple vision processing system.
phSimpleVision internal design
Internal to the class there exists one capture class that provides input to two display classes and one pipeline class. The pipeline class supplies input to two display classes, too. The capture class (derived from a phImageCapture class) opens a connection to a device capabale of capturing images and continually captures images to supply to the display and pipeline classes. The pipeline class (phPipeline) allows one to add a series of filter processing classes (derived from phFilter) that will be run on the images that are supplied from the capture device.
The two displays (derived from the phDisplayInterface class) provide the ability to view images either on a local machine or across the network. When one is trying out the phSimpleVision code on a desktop, it makes sense to use facilities which will provide the best display performance locally. For this reason, the X11Display is used to display images when one is running locally. If one desires to run the vision system on a remote machine, for example a robot, the best performance is acheived by compressing the image being displayed and then sending it across the network (as opposed to using the X protocol to remote display uncompressed images). For these reasons, two display classes are provided for each output within the phSimpleVision class: from the capture device and the pipeline class.
phSimpleVision example overview
There are currently 2 phSimpleVision examples. One for use with robots that have Pyro & PTZ capability and the other is a C++ example to demonstrate the features of phSimpleVision. The examples are located in the Phission example directory which can vary depending on where Phission is installed and whether you're using code checked out from CVS. If it is the former of the two, then you'll want to look for the examples in the installation directory: /usr/local/phission/examples/ or /usr/phission/examples/. Otherwise, the examples will be in in the directory 'examples' under the CVS checkout directory. All pyro examples will be located in examples/pyro and the C++ examples will be located in examples/cpp.
This example is located in phission/examples/cpp/phSimpleVisionTest. If you're working out of a Phission installation you might want to copy this directory to your own private working directory. See the next paragraph for how to copy an example to your own directory. If you own the directory and want to make changes directly in it or just want to run the examples then you can simply 'cd' to that directory. Once you have a working example directory you can run 'make' to build the example. The executable will be placed into the bin/ directory of the current directory.
To copy the phSimpleVisionTest directory to your home directory, the following procedure should be followed.
$ cp -pR /path/to/phission/examples/cpp/phSimpleVisionTest ~
$ cp /path/to/phission/examples/cpp/Makefile.example ~/phSimpleVisionTest
The phSimpleVisionTest has the following command line arguments available for use:
Add a method to add a custom filter into the pipeline
Add a print method to print out what the system looks like
Add the optional parameters to the filter methods to allow control of the filter parameters instead of using just the defaults.
This is a Pyro example that uses a Finite State Machine to control the robot and makes use of the Phission phSimpleVision class for PTZ training and tracking of
To copy the StateBasedSimpleVision.py example for your own use:
cp -pR /path/to/phission/examples/pyro/brains /home/username/etc
To simply run that example without modification:
$ cd /path/to/phission/examples/pyro/brains
$ pyro -r Aria -b StateBasedSimpleVision.py
phSimpleVision API - Display settings:
API Note (Displays):
For all the display methods, 'which' refers to which output the particular display is connected to.
Valid values include:
The short forms:
Performance Note (Displays):
A very useful feature is the ability to enable and disable the displays. A display is really only needed during debug or experimentation time. When it comes time to cut down on the amount of resources the system is using (to get better performance) the displays can all be disabled right before the main loop has been entered and all setup is done.
The NetDisplay functionality is disabled by default. You must enable it with one of the NetDisplay control methods. The NetDisplay's default port is 44444 for the Capture output and 44445 for the Pipeline output. There is a NetClientTest program availible for monitoring the output from a NetDisplay class and can be located in either "/usr/local" or "/usr" (depending upon installation):
If the executable isn't already built, you can copy the directory to your home directory using the following steps:
cp -pR NetClientTest ~/phission_examples
cp Makefile.example ~/phission_examples
The example requires the Makefile.example file to be one directory above (unless one wants to edit the Makefile in the NetClientTest directory to point to a different directory).
When the NetClientTest example is built, it can be executed as follows (from the NetClientTest directory):
./bin/NetClientTest 127.0.0.1 44444
phSimpleVision API - System control methods:
The vision processing system is not always running when an instance of the phSimpleVision class exists. Internal to all the Phission code exist many separate threads of execution to perform the many separate tasks required for image capture, processing and displaying. Before the system can do any processing it must be started and will then run concurrently with the normal application. Data and settings can be altered or retreived most the time while the threads are executing because careful care has been taken in implementing thread-safety within Phission. phSimpleVision takes care of calling the methods that are responsible for starting up the Phission vision processing system in three methods:
phSimpleVision API - Singular filter control methods:
These methods will run the respective filters by themselves within the pipeline. phSimpleVision handles all necessary pipeline accessing to remove the old filters and insert the new one.
phSimpleVision API - Histogram color training methods
phSimpleVision API - Internal object access methods
The following methods will return pointers to instances of internal objects contained in the phSimpleVision class. Sometimes the control methods provided by phSimpleVision just aren't enough and one needs more control. These should only be used if you know how to use the class objects being returned.
The interfaces to the following objects can be found in header files where Phission is installed. This is usually either going to be "/usr/phission/include" or "/usr/local/phission/include". Until the rest of the documentation for these objects is completed, it is suggested you look into the header files. Most the methods are self explainatory and there may be some enlightening comments, if any at all.
phSimpleVision example code
The examples are locate where Phission is installed. This directory will usually be "/usr/phission/examples" or "/usr/local/phission/examples".
phSimpleVision C++ examples
This demos all the singular filters when supplied the '--demo' parameter and will run the 'train' and 'track' filters otherwise. Pass it '--help' or see the source file for more command line switches.
phSimpleVision Pyro examples
This demonstrates a pan/tilt camera tracking a color that is trained on. When using Pyro, the command interface will accept 'self.dotrain = 1' to allow training without needing to press the bumper on the robot. This also demonstrates the use of a State Based brain for color training tracking and searching.
You'll likely want to run this program as such: pyro -r Aria -b StateBasedSimpleVision.py
The phSimpleVisionDemo example demonstrates all the singular filter methods of the phSimpleVision class. Closing the display that contains the output of the pipeline will cycle through the filters. This is a very convenient demo because it has two displays side by side to show the original live image and the output image. When using Pyro, the command interface will accept 'self.switchfilter = 1' to cycle through the filters, too.
The phSimpleVisionTest example demonstrates training and tracking functionality of the phSimpleVision class without the robotic requisites. Closing the Pipeline output display will toggle between the train and track methods. When using Pyro, the command interface will accept 'self.switchfilter = 1' to toggle between train and track methods
Definition at line 423 of file phSimpleVision.h.
Public Member Functions
|int||setPath (const char *path)|
|const char *||getPath ()|
|int||setChannel (uint32_t channel)|
|int||enableNetDisplay (int which=-1)|
|int||disableNetDisplay (int which=-1)|
|int||isNetEnabled (int which=-1)|
|int||setPort (uint32_t port)|
|int||enableDisplay (int which=-1)|
|int||disableDisplay (int which=-1)|
|int||isDisplayEnabled (int which=-1)|
|int||add (uint32_t nFrames=2, uint32_t frame_step=1)|
|int||train (int colorspace=-1, int add_or_set=-1)|
|int||trainSet (int colorspace=-1)|
|int||trainAdd (int colorspace=-1)|
|int||setColor (phColor color)|
|int||setThreshold (phColor threshold)|
|int||applyTraining (int add_or_set)|
|int||track (int colorspace=-1)|
|uint32_t||getBlobCount (uint32_t minsize)|
|blob_Filter *||getBlobFilter ()|
|phBlobData &||getBlobData ()|
|histogram_Filter *||getHistogramFilter ()|
|phHistogramData &||getHistogramData ()|
|phSystem *||getSystem ()|
|phImageCapture *||getCapture ()|
|phPipeline *||getPipeline ()|
|phDisplayInterface *||getDisplay (int which)|
|NetDisplay *||getNetDisplay (int which)|
Private Member Functions
|int||switchToFilters (int *indexes, int count)|
|int||switchToFilter (int index)|
|int||private_tracking (int which)|
|int||private_training (int which)|
|Copyright (C) 2002 - 2007||
Philip D.S. Thoren ( firstname.lastname@example.org )
University Of Massachusetts at Lowell