Andrew Birkett's nobugs.org
There doesn’t seem to be any tutorials which describe the process of creating a visualization plugin for xmms. I decided to create a funky 3d visualization plugin and keep notes about the process of making a fully fledged xmms plugin. I was inspired to write this tutorial after seeing the xmms-nebulus plugin, and I’ve used the nebulus sources as a guide and copied their automake magic wholesale. Feel free to use as much (or as little) of the code in this tutorial to make your own plugins. If you do write your own plugin, it’d be nice if you include a link to this tutorial in your documentation.
Before you get started, you need to make sure you have the relevant packages installed. You need gcc, automake/autoconf to build everything. Plus you need the development packages for glib, xmms, gtk and sdl, otherwise you won’t have the required headers in /usr/include. Plus, you should check that you can run an opengl application (such as glxgears) to make sure X11 knows about OpenGL.
Thanks to all the people who have offered feedback on this tutorial, including Eddy Nijland, Shirish Doshi, Gareth Williams, David Fischer and Björn Krüger. Much appreciated. :-)
XMMS plugin are compiled to shared libraries. If you look in
/usr/lib/xmms/Visualization you’ll see a number of libsomething.so files which correspond to the list of plugins you can see inside xmms. Building shared libraries isn’t the easiest thing in the world, so we’ll use the GNU build tools (automake/autoconf/etc) to do all the hard work for us.
I’ll be creating a plugin called xmms-nobugs, so start by creating a top-level directory called xmms-nobugs.
bash-2.05a$ mkdir xmms-nobugs
bash-2.05a$ cd xmms-nobugs
A plugin for XMMS has to do only one thing – export a function called “get_plugin_info” which returns a pointer to a VisPlugin struct. This struct is defined in the header <xmms/plugin.h>. It contains mostly pointers to function which will get called when useful events happen, such as playback starting.
So, without further ado, let’s create main.c which does the magic. It’ll just print a message to the console when an event happens. It’s about the most basic xmms plugin possible!
#include <stdio.h>
#include <xmms/plugin.h>
#include <string.h>
// Forward declarations
static void nobugs_init(void);
static void nobugs_cleanup(void);
static void nobugs_about(void);
static void nobugs_configure(void);
static void nobugs_playback_start(void);
static void nobugs_playback_stop(void);
static void nobugs_render_freq(gint16 freq_data[2][256]);
// Callback functions
VisPlugin nobugs_vtable = {
0, // Handle, filled in by xmms
0, // Filename, filled in by xmms
0, // Session ID
"Nobugs Plugin", // description
0, // # of PCM channels for render_pcm()
1, // # of freq channels wanted for render_freq()
nobugs_init, // Called when plugin is enabled
nobugs_cleanup, // Called when plugin is disabled
nobugs_about, // Show the about box
nobugs_configure, // Show the configure box
0, // Called to disable plugin, filled in by xmms
nobugs_playback_start, // Called when playback starts
nobugs_playback_stop, // Called when playback stops
0, // Render the PCM data, must return quickly
nobugs_render_freq // Render the freq data, must return quickly
};
// XMMS entry point
VisPlugin *get_vplugin_info(void)
{
return &nobugs_vtable;
}
static void nobugs_init(void)
{
printf("Nobugs plugin: Initializing\n");
}
static void nobugs_cleanup(void)
{
printf("Nobugs plugin: Cleanup\n");
}
static void nobugs_about(void)
{
printf("Nobugs plugin: About\n");
}
static void nobugs_configure(void)
{
printf("Nobugs plugin: Configure\n");
}
static void nobugs_playback_start(void)
{
printf("Nobugs plugin: Playback starting\n");
}
static void nobugs_playback_stop(void)
{
printf("Nobugs plugin: Playback stopping\n");
}
static void nobugs_render_freq(gint16 freq_data[2][256])
{
printf("Nobugs plugin: Render Freq\n");
}
We have a source file, but we need to find a way to compile it to a shared library. We’ll use the gnu build tools to do this. In short, we have to write “Makefile.am” and “configure.in” by hand and the tools will do the rest for us..
Firstly, create “Makefile.am” which contains the following. This will also link in the SDL and OpenGL library. These aren’t needed yet, but we’ll use them later.
lib_LTLIBRARIES = libnobugs.la
libdir = @XMMS_VISUALIZATION_PLUGIN_DIR@
CFLAGS = -Wall @XMMS_CFLAGS@ @CFLAGS@ -I$(top_builddir) -I$(top_srcdir) \
@GTK_CFLAGS@ @SDL_CFLAGS@
LIBS = @XMMS_LIBS@ @SDL_LIBS@ @GL_LIBS@
libnobugs_la_LDFLAGS = -module -avoid-version
libnobugs_la_SOURCES = main.c
Autoconf allows us to setup builds on various platforms (linux, bsd, mac etc) without doing too much horrible #ifdef stuff. Our configure.in file will check for required libraries (xmms, glib, gtk, sdl) and work out where the OpenGL headers are on the current platform.
Create “configure.in” which contains the following:
dnl Process this file with autoconf to produce a configure script.
AC_PREREQ(2.53)
AC_INIT(main.c)
AC_CANONICAL_TARGET([])
AM_INIT_AUTOMAKE(xmms-nobugs, 0.1.0)
AM_CONFIG_HEADER(config.h)
AM_DISABLE_STATIC
AC_PROG_CC
AC_PROG_CPP
AM_PROG_CC_STDC
AC_HEADER_STDC
AC_PROG_INSTALL
AM_PROG_LIBTOOL
AC_PATH_X
AM_PATH_GLIB(1.2.2,,AC_MSG_ERROR([*** GLIB >= 1.2.2 not installed - please install first ***]))
AM_PATH_XMMS(1.2.4,,AC_MSG_ERROR([*** XMMS >= 1.2.4 not installed - please install first ***]))
AM_PATH_GTK(1.2.2,,AC_MSG_ERROR([*** GTK+ >= 1.2.2 not installed - please install first ***]),gthread)
AM_PATH_SDL(1.2.0,,AC_MSG_ERROR([*** SDL >= 1.2.0 not installed - please install first ***]))
LIBS_save=$LIBS
LIBS="$LIBS $GTK_LIBS"
MATHLIB="-lm"
AC_PATH_X
AC_PATH_XTRA
if test x$have_x = xyes; then
CFLAGS="$CFLAGS $X_CFLAGS"
SYS_GL_LIBS="$X_LIBS -lGL -lGLU"
else
SYS_GL_LIBS="-lGL -lGLU"
fi
dnl Check for OpenGL
AC_MSG_CHECKING(for OpenGL support)
have_opengl=no
AC_TRY_COMPILE([
#include <GL/gl.h>
#include <GL/glu.h>
],[
],[
have_opengl=yes
])
AC_MSG_RESULT($have_opengl)
if test x$have_opengl = xyes; then
CFLAGS="$CFLAGS -DHAVE_OPENGL"
GL_LIBS="$SYS_GL_LIBS"
else
AC_MSG_ERROR(Unable to find OpenGL headers and libraries)
fi
AC_SUBST(GL_LIBS)
CPPFLAGS=$CPPFLAGS_save
LIBS=$LIBS_save
AC_OUTPUT(Makefile)
Now that we have Makefile.am and configure.in, we can utter the magic incantations to get us ready to compile! First we run “aclocal”, “autoheader” then “automake -a”.
bash-2.05a$ aclocal
bash-2.05a$ autoheader
configure.in:6: warning: do not use m4_patsubst: use patsubst or m4_bpatsubst
configure.in:79: warning: do not use m4_regexp: use regexp or m4_bregexp
autoheader-2.53a: `config.h.in' is created
bash-2.05a$ automake -a
automake-1.5: configure.in: installing `./install-sh'
automake-1.5: configure.in: installing `./mkinstalldirs'
automake-1.5: configure.in: installing `./missing'
configure.in: 578: required file `./ltmain.sh' not found
automake-1.5: Makefile.am: installing `./INSTALL'
automake-1.5: Makefile.am: required file `./NEWS' not found
automake-1.5: Makefile.am: required file `./README' not found
automake-1.5: Makefile.am: installing `./COPYING'
automake-1.5: Makefile.am: required file `./AUTHORS' not found
automake-1.5: Makefile.am: required file `./ChangeLog' not found
automake-1.5: configure.in: installing `./depcomp'
Automake complains about some missing files which are required in all GNU packages, but the “-a” option makes it create suitable files for us. We also need to create “README”, “AUTHORS”, “NEWS” and “ChangeLog” too, but they can remain empty for now. Finally we run “autoconf” which generates the familiar “configure” file which we all know and love.
bash-2.05a$ touch README AUTHORS NEWS ChangeLog
bash-2.05a$ autoconf
Some people have found they get a “can’t find libtool” error. I don’t know why I didn’t have this problem, but it can be resolved by running “libtoolize” to generate the required files.
We’re done! That was easier than I expected. We can now run “./configure && make” to build our plugin. Then “make install” will copy it into the xmms directory, and “make uninstall” will remove it. What’s more “make dist” will build a tarball ready for distribution.
Start up xmms from a shell, and then you’ll see our new plugin in the list of visualization plugins. When you enable it, you’ll see messages in your shell window. If you start playing music you’ll see that our render_freq() method get called lots during playback. That’s where we’ll get the information for doing sound-to-light from.
bash-2.05a$ xmms
Nobugs plugin: Initializing
Nobugs plugin: Playback starting
Nobugs plugin: Render PCM
Nobugs plugin: Render Freq
Nobugs plugin: Render PCM
Nobugs plugin: Render Freq
Yay, well done! We have created an basic xmms plugin which compiles and runs correctly. You can run “make dist” and give a copy of the tarball to all your friends so they can marvel at your coding skills.
The most important function in main.c is this guy:
void nobugs_render_freq(gint16 freq_data[2][256])
This function is called when xmms has some audio data ready. It is passed information about the frequencies in the audio data, with each channel being split into 256 frequency bands. There’s another function which xmms can call (but we don’t use it here) called render_pcm() which is given the raw waveform.
This function must return as quickly as possible. This is not the place to do intensive graphical computations! There’s time to do simple calculations, but if you spend too long then you’ll start getting pauses in the audio.
If we can’t do heavy processing in render_freq(), we’re going to have to spawn a second thread and let it process the data. Our worker thread will execute in parallel with the main xmms thread, and we’ll get uninterrupted audio.
SDL (Simple DirectMedia Layer) provides an abstract view of multimedia hardware, so that we can write multimedia apps and run them on linux, mac or windows. We’ll use their thread implementation to create our worker thread. Later, we can use more of SDL to do cool visualization stuff.
Let’s extend main.c to create a worker thread when the plugin is initialized, and kill the thread when the plugin is told to cleanup. The worker thread will spend its life printing a message every second. We will also need to have a flag so we can tell the thread when it’s meant to stop so that we can exit cleanly.
Add the following to the top of main.c:
#include <xmms/util.h>
#include <SDL/SDL.h>
#include <SDL/SDL_thread.h>
Then, just before nobugs_init(), add the following lines:
// Our worker thread
SDL_Thread *worker_thread;
enum { GO, STOP } thread_control;
void worker_func(void)
{
while (thread_control == GO) {
printf("Worker thread: Working away\n");
xmms_usleep(1000000);
}
printf("Worker thread: Exiting\n");
}
This is the code which the worker thread will run. The “thread_control” flag will be initially set to “GO”. The thread checks it once a second and exits if the flag is set to “STOP”.
Now we need new versions of our init() and cleanup() functions to deal with the SDL library and to create the worker thread. Change nobugs_init() and nobugs_cleanup() as follows:
static void nobugs_init(void)
{
printf("Nobugs plugin: Initializing\n");
if (SDL_Init (SDL_INIT_VIDEO) < 0) {
printf ("Failed to initialize SDL\n");
nobugs_vtable.disable_plugin (&nobugs_vtable);
return;
}
thread_control = GO;
worker_thread = SDL_CreateThread ((void *) worker_func, NULL);
}
static void nobugs_cleanup(void)
{
printf("Nobugs plugin: Cleanup entered\n");
thread_control = STOP;
SDL_WaitThread(worker_thread, NULL);
SDL_Quit();
printf("Nobugs plugin: Cleanup completed\n");
}
Notice that we call disable_plugin() if things go wrong. Our nobugs_init() can’t return a failure code so this is the only way we can signal to xmms that something has gone wrong. Also, we’re starting the thread in init() rather than in playback_start() to give us flexibility - we can do fun stuff when the music isn’t playing.
Now rebuild, install and run xmms. When you start the plugin you should see regular messages from the worker thread in addition to the messages which you saw before.
Now we want to start reacting to the music. We’ve already seen how our nobugs_render_freq() function gets called regularly with frequency information. However, this call occurs in the main xmms thread, and we want to be doing all the hard work in our worker thread.
So, we’ll create a buffer into which the main xmms thread will write frequency data, and the worker thread will read it from there. We need to use a mutex to ensure that they don’t access the buffer at the same time.
Extend the worker thread variables with a mutex and the shared buffer, as follows:
// Our worker thread
SDL_Thread *worker_thread;
enum { GO, STOP } thread_control;
SDL_mutex *mutex;
gint16 shared_freq_data[2][256];
Now extend our init() and cleanup() functions with a single line to create/destroy the mutex. We’ll also initialize the shared buffer.
static void nobugs_init(void)
{
printf("Nobugs plugin: Initializing\n");
if (SDL_Init (SDL_INIT_VIDEO) < 0) {
printf ("Failed to initialize SDL\n");
nobugs_vtable.disable_plugin (&nobugs_vtable);
return;
}
// Start worker thread
thread_control = GO;
mutex = SDL_CreateMutex();
memset(shared_freq_data, 0, sizeof(gint16) * 2 * 256);
worker_thread = SDL_CreateThread ((void *) worker_func, NULL);
}
static void nobugs_cleanup(void)
{
printf("Nobugs plugin: Cleanup entered\n");
thread_control = STOP;
SDL_WaitThread(worker_thread, NULL);
SDL_DestroyMutex(mutex);
SDL_Quit();
printf("Nobugs plugin: Cleanup completed\n");
}
Now we can copy the freqency data to the shared buffer with a new version of nobugs_render_freq():
static void nobugs_render_freq(gint16 freq_data[2][256])
{
int channel, bucket;
SDL_mutexP(mutex);
for (channel = 0; channel < 2; channel++) {
for (bucket = 0; bucket < 256; bucket++) {
shared_freq_data[channel][bucket] = freq_data[channel][bucket];
}
}
SDL_mutexV(mutex);
}
Notice how we grab the mutex before writing to the buffer and release it afterwards.
Now we can extend our worker thread function to use this frequency data. We’ll also shorten the delay in the worker thread to a tenth of a second.
void worker_func(void)
{
while (thread_control == GO) {
SDL_mutexP(mutex);
printf("Bass is %d\n", shared_freq_data[0][0]);
SDL_mutexV(mutex);
xmms_usleep(100000);
}
printf("Worker thread: Exiting\n");
}
Now rebuild, install and run xmms. You should see messages from the worker thread every tenth of a second. Start playing a song and you’ll see how the bass frequency changes. If you stop the song, it stays at the same value so we’ll update nobugs_playback_stop() to clear the array back to zeros:
static void nobugs_playback_stop(void)
{
printf("Nobugs plugin: Playback stopping\n");
SDL_mutexP(mutex);
memset(shared_freq_data, 0, sizeof(gint16) * 2 * 256);
SDL_mutexV(mutex);
}
It’s not yet a dazzling display of sound-to-light technology, but we’ve now completed all of the hard stuff. Our plugin compiles and runs in xmms, it launches a worker thread which it can cleanly shutdown, and the worker has access to the frequency data for the currently playing track. These are the major pieces of code which are required for all visualization plugins.
Now let’s make our plugin do something interesting. We’ll create a very rudimentary spectrum analyzer display using OpenGL. It’ll only be 2d, but it’ll show that all the OpenGL stuff is working correctly.
First of all, we need to include the OpenGL headers. Add the following lines to the top of main.c:
#include <GL/gl.h>
#include <GL/glu.h>
Now, let’s extend our code so that it creates a small window (using SDL) and sets it up ready us to perform OpenGL operations. We have to create the window in the worker thread, since we’re going to draw into it from there. We’ll also do a few OpenGL calls to set up a basic viewpoint. Add the following code to the start of worker_func():
// Create window for OpenGL output
if (!SDL_SetVideoMode(320, 240, 32, SDL_OPENGL)) {
printf ("Failed to create window\n");
nobugs_vtable.disable_plugin (&nobugs_vtable);
return;
}
glViewport(0,0,320,240);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45, 320/240, 1, 10);
glMatrixMode(GL_MODELVIEW);
glClear(GL_COLOR_BUFFER_BIT);
Now build, install and run xmms. You should see that our plugin now creates a small window, although it’s completely blank at the moment. If this stage doesn’t work, you should make sure that you have OpenGL properly installed on your machine – do other OpenGL plugins work?
Now let’s change our worker thread so that it draws into that window. We’ll keep things simple at first. We’ll draw a single vertical red rectangle which represents the loudness of the lowest bass frequency (ie. the number we’ve been printing to the console).
And replace the “while” loop in worker_func() with the following code. I’ve tweaked the delay so that it should update at around 70fps since that’s how fast your monitor redraws at.
while (thread_control == GO) {
static int rotation = 0;
float height;
// Get the bass amplitude
SDL_mutexP(mutex);
height = shared_freq_data[0][0] / 1024.0;
SDL_mutexV(mutex);
glLoadIdentity();
gluLookAt(0, 0, 5, 0, 0, 0, 0, 1, 0);
// Clear buffer
glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
glClear(GL_COLOR_BUFFER_BIT);
glColor3f(1.0f, 0.0f, 0.0f);
// Spin round a bit
glRotatef(rotation++, 0, 0, 1);
// Draw bar
glBegin(GL_POLYGON);
glVertex3f(-0.1, -1, 0);
glVertex3f(0.1, -1, 0);
glVertex3f(0.1, height, 0);
glVertex3f(-0.1, height, 0);
glEnd();
glFinish();
SDL_GL_SwapBuffers();
xmms_usleep(1000000 / 70);
}
Hey, now we’re starting to get somewhere! At this point, we’re starting to get into the world of pure OpenGL programming and all of the xmms-specific stuff has been handled. That seems like good place to end this tutorial.
In this tutorial we’ve seen how to create a skeleton xmms visualization plugin, using automake/autoconf to do all the hard work of building the shared library. We’ve seen how to create a seperate worker thread (so we don’t slow down the main xmms application) and how to pass data from the main thread to the worker. Finally, we’ve seen how to use SDL to create an output window and how to use OpenGL to display stuff in that window.
Hopefully this tutorial has provided you with all the information you need to develop your own xmms plugins. If you have any questions or if you’ve spotted a mistake, email andy@nobugs.org and I’ll do my best to answer them.