PyOpenGL too slow? Use C.

Recently I’ve been doing a lot of tinkering with the rendering routines in flamingo, which are still a bit primitive. The current way to render a bitmap image is to create a Sprite (with a bitmap loaded from the image module). Then the engine loops through all of the Sprites currently “alive” and then draws them with OpenGL primitives. There are definitely faster ways to accomplish the rendering, but I wanted to start with the simplest possible thing that would work.

(This is a slimmed down picture of what the engine is doing. There are also multiple “Screens”, so there is the potential to draw a single Sprite more than once. Currently the engine just loops through all of the Sprites for each Screen. There are better ways to do this, they will be implemented in the future.)

I was using PyOpenGL 3 to draw the primitives, but it was unexceptably slow. PyOpenGL_accelerate barely helped the problem. Even with only 3 Sprites the engine was running around 80 frames per second. Adding more Sprites slowed it down to around 20. (My old laptop is partly to blame for this.)

I experimented with Display Lists. They offered only a slight increase in speed. I implemented a method to cull the Sprites that had no chance of being drawn that frame (they were outside of all of the Screens). This didn’t help the speed much because there weren’t any Sprites not being drawn in my test (tests/flamingo_test.py).

After my options were exhausted (for the most part), I decided to do the same thing I did for the fmath module: I ported the drawing functions to C. I had planned to just call the PyOpenGL functions from C, but I decided to experiment. What I found was fairly surprising (to me) and pretty awesome. I’m documenting it here for others in the same situation.

Turns out you can include the OpenGL C header files to your extension module, link the OpenGL libraries to the extension module, call the C OpenGL routines, and, (if you tell Pygame to set up a OpenGL window), you can render directly to the Pygame window. You can also mix these C OpenGL calls with PyOpenGL calls with no fuss.

(From what I understand, this works because PyOpenGL and Pygame share the same rendering context. Loading the same shared library and not changing the context will result in OpenGL rendering to the same context.)

The results: much faster rendering without any headache. Here’s how to do it.

First off, you need to tell Pygame to set up an OpenGL window, which is very easy:


pygame.display.set_mode((800, 600), pygame.constants.OPENGL | pygame.constants.DOUBLEBUF)


Then I did the initial OpenGL setup with PyOpenGL calls (it’s convenient to do it right after creating the window, and that was called from Python anyway).

Then, when you create your extension module, be sure to include the header files:

#include <gl/gl.h>
#include <gl/glu.h>    // You may not need this
#include <gl/gle.h>    // Or this

Then, just call whatever OpenGL routines you need:

double hw = 50.0;
double hh = 20.0;

glBegin(GL_QUADS);

glTexCoord2f(0.0, 0.0);
glVertex2f(-hw, hh);
glTexCoord2f(0.0, 1.0);
glVertex2f(-hw, -hh);
glTexCoord2f(1.0, 1.0);
glVertex2f(hw, -hh);
glTexCoord2f(1.0, 0.0);
glVertex2f(hw, hh);

glEnd();

And the final step is linking the actual OpenGL libraries to the extension module when you compile it. Thankfully, distutils makes this very easy:

from distutils.core import setup, Extension

module = Extension('module', libraries = ['opengl32'], sources=['module.c'])
## You'll need to add 'glu32' and 'gle32' if you used those in your module

setup(ext_modules = [module])


And that’s all you need to do. Your mileage may vary depending on your PyOpenGL version, the OpenGL library version, your OS, etc. (You may need to specify ‘library_dirs’ in your setup script. The relevant Documentation is here).

The engine only experienced a slight increase from this, but I will implement more speed ups soon (Display Lists, using arrays instead of lists for Polygon types, porting image module to C, etc). Expect code updates soon.

Boring Status Update 1

Sorry there haven’t been updates in a while, I’ve been bogged down with schoolwork. I haven’t given up this project, however, and there are some great new updates and ideas floating in my head.

The fmath module was performing extremely slow, so I decided to rewrite it in C. There will be other modules written in and ported to C later, so this was some good practice. The port isn’t completely done, though; There is support for Vectors and Polygons, but not Rects, and there is no documentation yet. I’ll probably need to optimize it a bit more. The demo doesn’t work correctly yet with the C module because Rects aren’t implemented yet, but you can still build it by running this in the installation directory:

python setup.py build_ext --inplace

There won’t be more updates until at least a few weeks from now. After finals I’ll have more time to work on the project.

Never Trust A Software Developer

I’ve learned quite a lot developing Flamingo so far, though there isn’t a lot to show for it yet. Near the top of that list is that Software Developers lie (just take a look at 3dRealms’ track record). “Mostly Done” really means it’s about time for a rewrite in the software world, which is the case for the Flamingo math module (it’s named fmath so as not to override the built-in math module in Python).

The reason for the massive fmath refactoring is mostly lack of foresight on my part. The old version was very rigid and concrete, it was hard to get much done, and it was slow. The new version supports the abstract notion of Coordinates, Polygons, and Matrices, making it much more flexible and extensible. As a result you can now use tuples, lists, the concrete Vector class, or your own Coordinate class to show coordinate points. The same is true for Polygons (and, by extension, the Rect class which will be the basis of much of the rest of the engine). Matrices, however, are best handled by numpy. The wonders of Cython will also be taken advantage of in fmath and all of Flamingo.

This refactoring has gotten me thinking a lot about the rest of Flamingo and how it’ll be pieced together. I now know that I want it to be as easy to extend as possible, making it useful for people other than myself. For example, I need a way to make it easy to roll in a new audio engine or sprite system, which I suspect developers would like to extend. I’ll have to draw a line somewhere, though. What sections do I make concrete? What parts don’t need custom functionality by someone? Striking a balance poses an interesting problem.

Next up after the final touch-ups on fmath is the general rendering system, which will take advantage of OpenGL. I’m not going to go too far into this because I want to get started on the audio component soon (experience with Opossum tells me that it’ll get pushed to the end, at which point it’ll be too hard to integrate into the rest of Flamingo).

Source Updates
The repository has been updated so you can download the trunk without adding the installation path to PYTHONPATH (which is much more convenient ). The source is located here. The new fmath is not there yet, but it will be shortly.

Flamingo Update: fmath (mostly) done

Sorry about the lack of updates, I’ve been swamped with school. Don’t expect anymore soon, I’ve got finals next week. But after that I’ll be an evil little code monkey for two weeks (what else would I do with that Christmas break?).

In the spare moments I have had, I’ve churned out some very useful stuff: the fmath module. It has all the usual math functions a 2d game could ever need (distance, distance squared, polyline simplification, etc) and a 2d Vector class, which is invaluable for the engine.

Head on over to the repository to check out the latest version. The library won’t work correctly if you just download everything to the installation directory. You need to add the installation directory to PYTHONPATH or navigate to the directory (usually Python26\Lib\site-packages\flamingo) and type:

svn checkout http://flamingoengine.googlecode.com/svn/trunk/flamingo "" flamingoengine-read-only

I should probably setup the repository more conveniently in the future, but that’s for another time.

Flamingo – What’s the Deal?

(This post is rather old, and some of the information is wrong due to changes in the application. The general idea is still the same. For specific details, refer to the documentation (after it is written, of course).)

The reasoning behind the switch from Opossum to Flamingo is pretty simple. Think about Opossums. They’re hairy, smelly, disgusting, and unwanted. That’s how I felt about my engine. The more I added the more  locked in it became, the opposite of how I envisioned it at the start. I needed a clean slate. I needed something elegant, something unconfined, something faster, something pink.

At it’s heart Opossum was a sketchpad. It was a test. And now that I’ve learned what does and doesn’t work, I’m ready to build the real thing. Extensive planning has gone into the project already. Here’s a summary of how things are going to work for the most part in the final product:

Coordinates

There are two coordinate systems in Flamingo: the rendering coordinate system and the physics coordinate system used by box2d. The first step to understanding how everything fits together is figuring out how the two coordinate systems work together.

Opossum used native SDL rendering which made mapping coordinates a mess. (0,0) was in the top-left corner of the screen, so all of the physics had to be done in the IV quadrant and then the y-value had to be flipped for each entity before rendering. This caused a lot of headaches.

Flamingo will be migrating to OpenGL rendering. Luckily (0,0) is on the bottom-left of the screen in OpenGL, so most of the physics and all of the rendering coordinates will be positive.

In addition there are World coordinates and Local coordinates. World coordinates are relative to the origin and local coordinates are relative to an entity’s central point.

These coordinate systems are combined to form RW, RL, PW, and PL.

Sprites

Next is the concept of a Sprite, which I’m sure you’re all familiar with. A Sprite is simply an image that is blitted (drawn) to the screen. Sprites are grouped together in SpriteGroups and are all blitted together. Every Sprite instance is in at least one Master SpriteGroup. Sprites are positioned with RW coordinates.

Objects (subclass of Sprites)

An Object is a sprite that updates it’s position and rotation based on a box2d body (If you are unfamiliar with Box2d, I suggest reading the wiki). These can be either rectangular or circular and use PW coordinates.

Compound Objects

A Compound Object is essentially a collection of Objects that act together as a unit. They can be attached with joints or not.

Viewport

A viewport is a rectangular or circular area in RW coordinates. These are the “cameras” that let you see the world. Whatever is inside of these areas is blitted to the screen. These can include custom drawing functions to make things like Radars.

Map

A map is a collection of images, sounds, and scripts all bundled together with the MapEditor. These are saved as .fmap files and loaded directly into the Game. All of the resources are tied up in the map file, so distribution is extremely easy.

This is one of the core aspects of Flamingo. All of the scripts are offloaded into the map files themselves. All you have to do is load map and the scripts are loaded with it. Doing the scripts in this way makes development much easier; you no longer need to add a bunch of specific map code to the core source files themselves, eliminating all the headache associated with it. In addition it makes it very easy for the players to create and distribute their own maps.

Character

A character is a specialized Compound Object with sounds, scripts, and animations. They are created with the CharacterBuilder, saved as .fchar files, and loaded into the Game. Characters follow the same script philosophy as Maps.

Opossum Gets a Makeover

Nearly 6 months ago I started working on an open-source, 2d game engine with 3 things in mind:

1. Written in Python
2. Easy to Understand
3. Versatile

What I came up with I called the Opossum Engine. Though far from being done, it satisfied all of the requirements above. The problem was it was a bit bulky and some things clearly needed an overhaul. Much of the engine was (thoughtfully) hacked together, and it accumulated quite a bit of cruft. In addition, I was stuck with design choices made in the beginning of the project (like native SDL rendering instead of the faster OpenGL), which I regretted later. I decided it was time to scratch the idea and start over completely, using what I’ve learned so far in the making of Opossum.

And thus Flamingo was born. It’s being built from scratch, based on the original work from Opossum. It will be designed with speed and simplicity in mind, while remaining as versatile as possible. Many features and concepts will remain the same in this change. A more substantial overview will be posted in the future, but for a quick overview:

• Python (based on pygame)
• OpenGL rendering
• Extensive sprite system
• 2d Physics based on Box2d
• Versatile Audio Mixer with unlimited number of simultaneous sounds
• Advanced Maps (levels) and Characters (with physics-based animation) built in specialized editors
• Scripts in Python, which are mostly bundled with the Maps and Characters

Flamingo obviously has quite awhile to go before being released, but initial development should be rapid.

The original Opossum Engine will be released soon, though it is extremely far from being production ready. Check here or the Pygame website for the release details.