Generator multimedia project
4. MAX/MSP/NATO/SPAT DEVELOPING ENVIROMENT:
4.1. Introduction:
I have been working with MAX/MSP software since 1999 and in 2001 added NATO to my software development toolkit. This year I worked with SPAT and a MOTU 828 multi-channel audio interface.
During the last four years I collaborated with several individuals from the MAX/MSP/NATO community, such as: Johnny deKam (USA), Brian Kane (USA), Brian Ziffer (USA), Jash (Switzerland), Torsten Lauschmann (Scotland), Iko (France) and Carlos Guedes (Portugal) exchanging information and developing individual and collective projects during the last two years.
Some of this collaborations resulted in real life applications and projects, for example: I’ve been beta testing and inputting ideas and features on VDMX, Johnny deKam video mixer software developed in MAX/MSP/NATO.
This year, I developed some applications in MAX/MSP/NATO, some for my personal use and others for other people’s use. For example a kaleidoscope software developed specially for a concert performed by Lee Ranaldo and Rafael Toral, and joined by Leah Singer (16mm film manipulation) and Paulo Feliciano (software operation) in the Serralves Museum. Kaleido (the name of the software) used the input of a webcam and outputted the incoming video signal transformed into a kaleidoscope, providing interesting video effects displayed in TV sets placed throughout the stage. For my personal use, I’ve developed some applications related to video mixing which I use in my DJ/VJ sets, it it’s curious to note that some exhibit somehow a generative behavior because they use audio as their main control data but transform video very freely with little intervention from me.
4.2. MAX:
MAX was initially developed at IRCAM by Miller Puckette, about 10 years ago, and has subsequently been taken over by David Zicarelli (first, through Opcode Systems and, more recently, through David’s own company, Cycling ’74.
MAX was conceived as a toolkit or ‘kitchen’ for electronic music (which, given the technology of the day, meant MIDI). It utilizes a graphical paradigm in which various “objects” (representing programming routines) are connected using virtual patchcords (indicating data flow). It’s sort of like building a flowchart, or working with an analog synthesizer. MAX comes with all sorts of basic objects to get MIDI information in and out, move numerical data around, transform the data in various ways and control some elements of the user interface. (If you don’t know what MIDI is, it’s basically control information for a wide variety of electronic devices, like sound synthesizers and lighting boards).
4.3. MSP:
A few years ago, Cycling ’74 introduced MSP, which extends MAX by adding audio capabilities into the same basic architecture. With MSP, MAX became a fully configurable software synthesizer, sampler and sound laboratory. As with MAX, functions are provided to get audio in and out (with support for numerous 3rd party sound cards), move audio data around and alter it with filters, numerical functions, whatever. MAX and MSP can interact, of course, which means that the control structures built in MAX can interface with the new audio objects – the audio can be controlled and affected by MIDI data.
4.4. NATO:
NATO.0+55+3d modular, authored by 0f0003.MASCHIN3NKUNST comprises a set of QuickTime externals for MAX. Which is to say, NATO.0+55+3d modular permits you to deal with any sort of QuickTime media (films, images, sound, QuickTime VR, QuickDraw 3D, Flash movies, etc.) from within MAX in the same fashion as you can deal with MIDI or audio data using the built-in functions or MSP. NATO.0+55 interfaces with MAX in the same manner that MSP does – seamlessly. MIDI and numerical data can be used to control any NATO.0+55 function. Because it’s MAX, you can build complex structures around your QuickTime data that permit control at whatever level you prefer.
4.5. SPAT:
Spat~ is a configurable real-time spatial processor integrating the
localization of sound events (directional sound processing) with room
acoustical quality (artificial reverberation). Since it is based on a
modular organization, it provides both a complete application and a library
of Max objects for real-time spatial processing of sounds.
A Spat~ processor receives sounds from instrumental or synthetic sources,
adds spatialization effects in real time, and ouputs signals for
reproduction on an electroacoustic system (loudspeakers or headphones). The
general approach taken in Spat~ can be characterized by the fact that it
gives the user the possibility of specifying the desired effect from the
point of view of the listener rather than from the point of view of the
device or process used to generate that effect.
4.6. Beyond MAX/MSP/NATO/SPAT:
Recently, several advances have been made in the MAX externals front, among them SoftVNS 2 and a complete suit of external dedicated to video processing and control from Cyclling’74 named Jitter. Probably GENERATOR will incorporate them in one way or another, mainly because of their performance enhancements with real time video processing and OpenGL.
4.6.1. SoftVNS 2:
SoftVNS 2 is a set of 98 external objects for MAX/MSP that allows you to process and display up to 640 x 480 color video at 30 frames a second in real-time.
All functions are coded for the velocity engine on the G4. This results in 4 x the processing speed of non-velocity engine code. And video can be processed in YUV mode as well as RGB. YUV yields twice the performance of RGB with no visible loss of quality. Add the two up and you get up to 8 times the video performance of non-Velocity Engine RGB code. (Note: SoftVNS does not speed up quicktime movie access itself)
4.6.2. Jitter 1.0:
Jitter is a set of video, matrix, and 3D graphics objects for the Max graphical programming environment. Jitter provides a set of new objects for describing and manipulating matrix data – any data that can be expressed in rows and columns, such as video and still images, OpenGL-based 3D geometries, as well as text, spreadsheet data, particle systems, voxels, or audio.
Helder Luis, 2002
