Table Top Framework Example Application

Building an Example Application

This tutorial gives a brief overview how to build the example application that comes with the buffer framework. It can also be used as a guideline to build other applications with the buffer framework.

It is very useful (if not even necessary) to have a good understanding
of the framework architecture from reading the information provided in the

Getting Started section.

Application Overview

The application is built on Trolltech's Qt and, thus, the main body consists of three Qt objects:

QApplication, which is the main application,
A main window that contains all the window-related application logic, and
A widget for rendering OpenGL content.

Certain information for both the main window and the rendering widget is defined in a configuration file which is read at run-time (this saves recompilation for minor changes).

The following figure gives an overview of the classes that are used in the example to built an application:

In the next sections, selected aspects of such an application are presented.

Feeding Input from the Application and/or an Input Toolkit into the Buffer Framework

Forwarding input from the application or an input toolkit into the framework is an important taskt. The buffer framework provides a simple and powerful interface to achieve this.

SMART DViT

The example application uses the SMART SDK to enable two touches on a SMART DViT display. Thus, we implement the following methods and pass the gathered information on to the framework:

void QTTabletopWidgetGL::OnXYDown(int x, int y, int z, int iPointerID)
{
getVis()->devicePress(x, (this->height() - y -1), z, translateSmartIPointerId(iPointerID));
}

void QTTabletopWidgetGL::OnXYMove(int x, int y, int z, int iPointerID)
{
getVis()->deviceMove(x, (this->height() - y -1), z, translateSmartIPointerId(iPointerID));
}

void QTTabletopWidgetGL::OnXYUp(int x, int y, int z, int iPointerID)
{
getVis()->deviceRelease(x, (this->height() - y -1), z, translateSmartIPointerId(iPointerID));
}

The last parameter of each method, translateSmartIPointerId(iPointerID), is the id of the interacting user. The framework expects here a number between 0 and the number of users minus one.

On the DViT board, we cannot really identify the user, but we get an id from the SDK, which we have to translate. The numbers used here are due to the SDK specifications.

unsigned int QTTabletopWidgetGL::translateSmartIPointerId(int iPointerId)
{
switch (iPointerId)
{
case 1:
return 0;
break;
case 257:
return 1;
break;
default:
return 0;
}
}

Trolltech Qt

To have the application also running on regular computers without input from a SMART board, we have to implement the regular input methods Qt provides. As before, we just pass the information on the framework, but we do not care about the user id here as there is only one input (mouse) used in this context.

void QTTabletopWidgetGL::mousePressEvent(QMouseEvent *e)
{
getVis()->devicePress(e->x(), (this->height() - e->y() - 1), 0, 0);
}

void QTTabletopWidgetGL::mouseReleaseEvent(QMouseEvent *e)
{
getVis()->deviceRelease(e->x(), (this->height() - e->y() - 1), 0, 0);
}

void QTTabletopWidgetGL::mouseMoveEvent(QMouseEvent *e)
{
getVis()->deviceMove(e->x(), (this->height() - e->y() - 1), 0, 0);
}

Creating the Visualization Content via the Builder Design Pattern

As described in the thesis on the buffer framework, the application content is created via the Builder design pattern.

Build Director

The Build Director -- here TabletopVisBuildDirector -- provides an interface to built complete visualizations with a single call of a method. This is achieved by accumulating multiple calls to the Builder interface. The example application can access this functionality by inheriting it from TabletopVisBuildDirector. Thus, TabletopVisBuildDirector is the place where the content of an application gets defined. Even several different definition are possible to change the content dynamically at run-time. An example would be the following method that creates the content the example application starts with.

TabletopVis* TabletopVisBuildDirector::createFirstDemo(TabletopVisBuilder* builder)
{
// the builder needs to have a Vis
if(builder->getVis())
{
builder->buildTextures(config->NUMBER_OF_TEXTURES, config->TEXTURES_SOURCE);
builder->buildComplexImages(config->IMAGE_LEAF_GL_BASE_WIDTH, config->IMAGE_LEAF_GL_BASE_HEIGHT, config->NUMBER_OF_OBJECTS, config->NUMBER_OF_TEXTURES);
builder->buildSimpleCurrentWithImages(config->CURRENT_GL_BASE_WIDTH, config->CURRENT_GL_BASE_HEIGHT, config->OBJECTS_IN_CURRENT_SIZE, config->OBJECTS_IN_CURRENT_SPEED, config->IMAGE_LEAF_GL_BASE_WIDTH, config->IMAGE_LEAF_GL_BASE_HEIGHT, config->OBJECTS_IN_CURRENT, config->NUMBER_OF_TEXTURES);
builder->buildStorageBin(config->BIN_GL_BASE_WIDTH, config->BIN_GL_BASE_HEIGHT, config->OBJECTS_IN_BIN_SIZE);

return builder->getVis();
} else {
return NULL;
}
}

This method loads the textures, creates images, an Interface Current with images inside, and a Storage Bin. The resulting visualization is returned the calling client code for further processing (see below). (It is important to understand that all the parameters for the methods are set via the read configuration file. This enables changing details without recompilation of the project.)

Using the Builder

After setting up, which content we want to create, this section deals with how to call the methods we just created. During initialization of the OpenGL widget we create the Builder and build an empty Vis for it.

// create a renderer-specific Builder
// if a Builder for a different renderer would be required
// it has to be created here
// this enables changing the renderer at run-time
// while still having access to the same Builder and Director methods
glBuilder = new TabletopVisBuilderGL();
// Vis has to be built in advance to enable setup
glBuilder->buildVis(config->NUMBER_OF_USERS, config->BUFFER_WIDTH, config->BUFFER_HEIGHT, this->width(), this->height());

After that, we do some configuration for hardware acceleration:

// hardware accelleration setup for the visualization
_mipMapping = _linearFiltering = _textureCompression = true;
_arbTextureCompression = false;
glBuilder->getVis()->enableHardwareOptimization(true);
glBuilder->getVis()->enableMipMapping(_mipMapping);
glBuilder->getVis()->enableLinearFiltering(_linearFiltering);
glBuilder->getVis()->enableTextureCompression(_textureCompression);
glBuilder->getVis()->enableARBTextureCompression(_arbTextureCompression);

Last but not least, we setup the visualization for the specific rendering API and call the method we defined above:

glBuilder->getVis()->initializeRenderer();
setVis(createFirstDemo(glBuilder));

The visualization is now set with the desired content and can be changed any time with a similar call.

The Render Loop

This part is described in greater detail in the thesis. Here just a brief look at important parts of the render loop.

The picking or dropping parts of the render loop have to be done for every user seperately. Therefore, we have to iterate over all users:

void QTTabletopWidgetGL::paintGL()
{

// ...

// iterate over all users
for (unsigned int i = 0; i < config->NUMBER_OF_USERS; ++i)
{
// if there was an active component,
// make it the last active one (active then NULL)
getVis()->resetActiveComponent(i);

GLubyte pixelColor[3] = {0,0,0};

The color picking for Components starts here:

// COLOR PICKING
// do painting (or picking)
// do we need to draw?
if ( getVis()->checkDoDrawingForUser(i) )
{
GLubyte pixelColor[3];
// this is important to do the expensive operations not too often
if (getVis()->getIsPicking(i))
{
glPushAttrib(GL_ALL_ATTRIB_BITS);
// render without the textures and only where the input is
glDisable(GL_TEXTURE_2D);
// important so we won't see the other Components moving without textures
glScissor(getVis()->getPosXToPaint(i), getVis()->getPosYToPaint(i), 1, 1);
glEnable(GL_SCISSOR_TEST);
// render Components for picking
getVis()->renderAllComponentsForPicking();
// finish the rendering (is important on quite fast machines,
// otherwise we read wrong stuff from the cursor)
glFinish();
// find pixel under mouse pointer
glReadPixels(getVis()->getPosXToPaint(i), getVis()->getPosYToPaint(i),
   1, 1, GL_RGB, GL_UNSIGNED_BYTE, pixelColor);
//glDisable(GL_SCISSOR_TEST);
//glEnable(GL_TEXTURE_2D);
glPopAttrib();
}

// let the Vis do the processing
// we don't use the information "wasSuccessful"
// but we could (this is only a hint)
bool wasSuccessful = getVis()->processSingleComponent(i, pixelColor);
}

The dropping of a Component is handled separately:

// handling where we dropped something
if( getVis()->checkWasDroppedByUser(i) )
{
// unfortunately, we have to render nearly all the stuff (actually, only the composites)
// but note: this should never happen the same time all the stuff above is rendered
// (because we already picked something) :-)

glPushAttrib(GL_ALL_ATTRIB_BITS);
// render without the textures and only where the input is
glDisable(GL_TEXTURE_2D);
// important so we won't see the other Components moving without textures
glScissor(getVis()->getPosXToPaint(i), getVis()->getPosYToPaint(i), 1, 1);
glEnable(GL_SCISSOR_TEST);
// render only the composites (components that can contain other components) for picking
// the user id is required here because we do not want to render the dropped component itself
getVis()->renderCompositesOnlyForPicking(i);
// finish the rendering (is important on quite fast machines, otherwise we read wrong stuff from the cursor)
glFinish();
// find pixel under mouse pointer
glReadPixels(getVis()->getPosXToPaint(i), getVis()->getPosYToPaint(i), 1, 1, GL_RGB, GL_UNSIGNED_BYTE, pixelColor);
//glDisable(GL_SCISSOR_TEST);
//glEnable(GL_TEXTURE_2D);
glPopAttrib();

// let the Vis do the processing
getVis()->processDroppedComponent(i, pixelColor);
}

In the end, we move the cursors according to the interaction and render everything for real:

// move the cursor to where we are
//(current position determined in Vis automatically based on code above)
getVis()->moveTouchIndicator(i);
}

// THE REAL RENDERING

}