1) download gem binaries for os x and gem source / only source in linux.
2) Create a project in /extra just like the other projects there.
3) modify the config files as specified in "README-EXTRA.txt"
4) run ./autogen.sh
5) run ./configure --enable-fat-binary=i386 --with-pd=/path (ignore the --enable-fat-binary=i386 flag in linux, just use it in recent versions of OSX)
6) cd into the project folder (gem/extra/pix_myexternal) and run make
7) put the project folder in the path of Pd.

Flotante – for Silent Drum, Percussion and Mallet Tracking – Jaime Oliver (2007-8)

Silent Construction 1 – for Silent Drum – Jaime Oliver (2008-9)

3 Environments – for Silent Drum – Jaime Oliver (2010) Download the patch

Silent Screen – for Silent Drum, Saxophones and Guitar – Aaron Einbond (2011)

Zero Morphism - for Silent Drum, Saxophones and Guitar - Alec Hall (2009/11)

Syncro-vox and Other Cheap Animation Techniques - for Silent Drum, Saxophones and Electric Guitar – Bryan Jacobs (2011)

Dirty, noisy, polluted - for Silent Drum, Saxophones and Electric Guitar - Paul Clift (2011)

Once the drum is ready, we need to setup tracking. If you are unsure if this will work in OS X or Linux, then you can also try this first. ¹

First you’ll need a copy of Pd or Pure Data for your operating system.

In the event that you are using the same camera that I am, the PS3eye, you will also need to make sure that you have the correct drivers for your operating system. A small tutorial on how to install these drivers is found here.

Tracking and sound patches are separate for processing reasons and to allow sending the tracking data to other programs such as Max/MSP or others. It is currently sending messages through netsend/netreceive.

Download the tracking patch for osx and linux here.²

Uncompress and open the main.pd patch. This will load a pre-compiled library of GEM as well as best_match and fill0s externals. If you have Pd pointing to another copy of GEM you need to disable it. If you’d like to have pix_drum in your own copy of GEM you can include the sources in the /pixes folder and recompile or get the latest GEM. ³

main.pd patch for video tracking

In the main.pd patch there is a select camera option. This is only helpful in OS X. It will open a dialog to choose the camera and frame rate, as long as you have the correct drivers for the PS3 EYE camera. In this dialog you should also select the 320×240 resolution.

In linux it should go directly to the PS3eye camera, if it is the only video device open. Size and frame rate must be set separately through modprobe.⁴

A window should be created automatically, but you can create a window in the “create window” dialog at the top.

Most video tracking parameters are already set in advance for 320×240 resolution. You will only need to adjust the threshold to your own lighting situation. The other variables are left there in case you want to explore what they do.

You’ll also need to adjust the bounds to the area of analysis desired. Right and left should reach the edges of the drum (of the open area). top should be right below the spandex head and bottom should be adjusted to the lowest the head can be pressed.

Once variables are set, press the “save” button to save the variables. They will be loaded automatically the next time you open the patch.

Once the receiving (netreceive) patch is open, press connect to send the data.

It is important to note that the camera must be correctly positioned before tracking. Ideally, the left and right extremes of the video shot must fit the edges of the drum. This will constraint the distance. The key point however is that the drum head must be flat. If it is pointed from below or from above, you will see a black oval; if it is perfectly positioned you’ll see a line. This constraints the angle. (a matter of perspective…)

The output of this patch is described here. You can also consult the pix_drum-help.pd patch in the /src folder.

Let me know if something is missing…

<< PREVIOUS: Hardware: Building

Notes:

In Construction…

In this post I will try to explain in as much detail as possible how to build a MANO Controller. The MANO Controller is an open source instrument and the code for pix_mano is released under the GPL license.

I run it in a 3.0ghz linux box (12MB Cache), running the camera at 75fps. Similar machines should make it…

Click on images in the tutorial to enlarge…

Hardware: Parts List

Hardware: Building

Software: Tracking

Software: Mapping Sounds

Performance Strategies

Compositions for the MANO Controller.

download the source code and patches for pix_mano. documentation is on the works… | bájate el código patches para pix_mano. documentación en proceso.

*********************************

Figure 1. One of the three sections

1) The first step is to cut the drum in 3 parts. The reason to do this is two-folded. On the one hand it needs to have one third of it completely open so as to let the camera see the shape of the spandex head when it deforms and the second is to make a device with which one can travel. If you bought a 14 inch drum, the circumference is about 44 inches. I cut mine in three sections of around 16, 15.5 and 12.5 inches. I left the 15.5 inch section out as the open section. In figure 1 you can see one of the sections.

The drum you bought should have included the body of the drum and two metal rims with their respective screws. It will also probably come with one or two drum heads which we will not use.

.

Figure 2. Attaching the two sections

2) the next step is to make two small holes (that coincide with the metal piece of fig 6 in the parts list) at the top, or what we want the top to be, so that they can be held together as seen in figure 2.

Attach them together with two screws and bolts. You can put some washers there if you want.

We should now have 2/3rds of a drum.

.

.

Figure 3. Bottom rim and screw.

3) Now we need to attach the lower rim to the 2/3rds of a drum we achieved in the previous step.

Use the rim and screws that came with the drum. The drum key will come in handy for this part. It should look like Figure 3. Note the rim is upside down. Regularly, rims are used to stretch a drum head, we on the contrary, are using it to get the drum to be compact/solid/strong after cutting it in pieces.

.

.

Figure 4. sawed spandex

4) The next step is to prepare the spandex head and the rim. I used two layers of spandex fabric and a thick elastic band. Choosing the spandex is not straightforward. I just went to a textile shop and chose one. The key is that it deforms easily, but recovers its flat shape very fast and doesn’t wear too much over time.

Figure 5. rim with spandex

Form a circle with the elastic band, then saw a fold of the two layers of spandex over it and saw them as shown in figure 4.

After sawing the spandex, attach it to the top rim with the nylon (I use it because it is resistant, but you can use something else). Make sure that the spandex head is stretched at the lower level of the metal rim as seen in figure 5.

I’ll post new pictures when i get some time to build a new head.

.

.

Figure 6. Top Rim

5)

The top rim needs to be adjusted to the drum body. Use the cut rod and nut numbered 2 in fig. 4 of the parts list.

.

After screwing the rod into the body of the drum, use a nut to set the level at which the rim will rest as shown in figure 6.1.

.

Fit the rim in the rods until it rests on top of the nut as seen in Figure 6.2.

.

Then adjust the rim with the nuts and cap (shown in figure 5 of the parts list) as seen in Figure 6.3.

.

The lower part of the rim needs to be fixed at the level where the body of the drum ends. The result should look like Figure 7.

Figure 7. modified drum

.

Figure 8. Paper and paperclips

6)

Once the structure is built, we need to prepare for tracking.

Get the white cardboard (it should be a flexible and light reflective thick white paper).

Cut it so as to cover all the interior of the drum.Once you have it cut, hold it to the body of the drum with paper clips as seen in Figure 8.

It is important to make sure that when you press the spandex, it doesn’t get stuck with the white cardboard/paper.

The result should now look like Figure 9.

Figure 9. white reflective inside

Figure 10. Two views of the Clamp in the Lower Rim.

7)

At this point the drum structure is ready.

We now need to attach the clamp and arm. the clamp must be attached to the lower metal rim as seen in Figure 10. I put it to a side to allow visibility by the audience, but, it can be set at any part of the rim. It is crucial that the clamp does not move at all.

.

.

Figure 11. Insert arm into Clamp

Once the clamp is tightly attached, we need to insert the arm into the clamp as seen in Figure 11.

The camera needs to be prepared in advance. Remove the base of that comes with the PS3eye camera. You can do this by removing the plastic covers that are on top of the screws, opening it up (carefully!), removing the base, and screwing it up again. You can see someone doing it in the embedded video below. We are not interested in removing the IR filter, just so you know how and what to expect when opening the camera without breaking it.

Use two nuts (that are the same size as the screw in the bracket) and stick them to the back of the camera. Use very strong glue, I use a strong weld. Superglue and the like won’t be enough.

Figure 12. Camera and Bracket

The camera then should look like it does in Figure 12.

Finally the camera should be fixed by screwing it to the bracket. We’ll calibrate and position it correctly in the tracking software section.

.

.

8)

We can now attach the drum to the stand and attach the lamp to the stand as well and we have all the hardware in place. Below are some images.

<< PREVIOUS: Hardware: Parts List

<< PREVIOUS: Hardware: Parts List —————————————————————— NEXT: Software: Tracking>>

//////////////////////////////////////////////////// LINUX \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

Linux drivers are available from http://kaswy.free.fr. In Fedora, I use the MT gspca modified driver V0.5 and compile it as explained in the driver’s files.

Essentially:

0. Download the driver from: MT gspca modified driver V0.5

1. cd to the main folder

2. type make

3. type make install

4. voilá

These steps have worked out of the box in fedora 10 and 11. You may have to fix things in other releases/platforms, for example, I’ve had a lot of trouble in Fedora 13. See http://nuigroup.org on the web to see some possible solutions.

There are references, an ubuntu tutorial and an explanation on how to load the driver here: http://kaswy.free.fr/?q=node/38#ps3

Note: to make GEM work with the ps3eye, it must be compiled against libv4l-dev or libv4l-devel, depending on your distribution. Just run yum install libv4l-devel in fedora. Then recompile GEM.

/////////////////////////////////////////////////////  OS X \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

I use the latest cvs build of the macam drivers. The regular driver won’t allow for higher (>30) frame rates.

To allow the driver to work with all applications, you need to put the file macam.component in /Library/Quicktime

In GEM, open the camera through the dialog message to pix_video (see pix_video-help.pd in the /help files)

\\\\\\\\\\\\\\\\\\\\\\\\\\ OTHER

If you’re looking for windows or if this is not enough information you should check out : http://nuigroup.org

Information:

The [tracks] object is designed to find the best continuity between indexed information. Usually tracking objects output data in order of appearance and need further logic to achieve continuity. Best continuity is defined as the smallest euclidean distance between two points in contiguous frames of time.

The graphic representation below is a one dimensional example:

In the graph, arrows show tracks with the best possible continuity.

A maximum distance determines if a datum will be kept in the same track. For example index 2 in t1 and index 2 in t2, have a distance of 24. if the maximum distance is 20, a new track will be created.

[tracks] will behave in the following way, for more information, see the help-file:

Information:

This object for Pd/GEM is now released as part of the extra folder in the latest GEM. If you download GEM, you should have it compiled for your platform. Epic Jefferson provides some advice on getting it working on windows.

Information

This object for Pd/GEM is now released as part of the extra folder in the latest GEM. If you download GEM, you should have it compiled for your platform. Epic Jefferson provides some advice on getting it working on windows. Menno Van der Woude reports developing an OpenFrameworks version.

Information:

This object for Pd/GEM is somewhat abandoned, but the principle with which it works is better than most color tracking algorithms when one wants to track pure colors, that is, the corners of an RGB cube space.

After several experiments with saturation, brightness, contrast and other tests, I discovered that the pixels that correctly tracked the color, were perfectly described by a plane such as the one in the RGB space image beside, leaving the top corner as our objective to track. Instead of this piramid, most video trackers look for a cube on the top red corner, allowing for pixels that are closer to grey than to red.

0.40 0.80 0.00