Posts: Cell

Past days' work

Been tracking and resolving a bug in Linux's SPU scheduler for the last three days, and fixed it just a moment ago! I'm happy and needed to mention this ;-)

More specifically, tracking it down was fairly easy using SystemTap and Paraver (getting the two to play well together was another source of headaches), but fixing it was the most complex thing due to deadlocks popping up over and over again. Sorry, can't disclose more information about it yet; want to think a bit more how to make this public and whether my fix is really OK or not. But be sure I will!

December 7, 2007 · Tags: cell, linux
Continue reading (about 1 minute)

Thanks, SystemTap!

I started this week's work with the idea of instrumenting the spufs module found in Linux/Cell to be able to take some traces of the execution of Cell applications. At first, I modified that module to emit events at certain key points, which were later registered in a circular queue. Then, I implemented a file in /proc so that a user-space application could read from it and free space from the queue to prevent the loss of events when it was full.

That first implementation never worked well, but as I liked how it was evolving, I thought it could be a neat idea to make this "framework" more generic so that other parts of the kernel could use it. I rewrote everything with this idea in mind and then also modified the regular scheduler and the process-management system calls to also rise events for my trace. And got it working.

But then, I was talking to Brainstorm about his new "Sun Campus Ambassador" position at the University, and during the conversation he mentioned DTrace. So I asked... "Mmm, that tool could probably simplify all my work; is it there something similar for Linux?". And yes; yes it is! Its name, SystemTap.

As the web page says, SystemTap "provides an infrastructure to simplify the gathering of information about the running Linux system". You do this by writing small scripts that hook into specific points of the kernel — at the function level, at specific mark points, etc. — and which get executed when the script is processed and installed into the live kernel as a loadable kernel module.

With this tool I can discard my several-hundred-long changes to gather traces and replace them with some very, very simple SystemTap scripts. No need to rebuild the kernel, no need to deal with custom changes to it, no need to rebuild every now and then... neat!

Now I'm having problems using the feature that allows to instrument kernel markers, and I need them because otherwise some private functions cannot be instrumented due to compiler optimizations (I think). OK, I'd expose those functions, but while I'm at it, I think it'd be a good idea to write a decent tapset for spufs that could later be published. And that prevents me from doing such hacks.

But anyway, kudos to the SystemTap developers. I now understand why everybody is so excited about DTrace.

December 2, 2007 · Tags: cell, linux, systemtap
Continue reading (about 2 minutes)

Hello world in Linux/ppc64

I'm decided to improve my knowledge on the Cell platform, and the best way to get started seems to be to learn 64-bit PowerPC assembly given that the PPU uses this instruction set. Learning this will open the door to do some more interesting tricks with the architecture's low-level details.

There are some excellent articles at IBM developerWorks dealing with this subject, and thanks to the first one in an introductory series to PPC64 I've been able to write the typical hello world program :-)

Without further ado, here is the code!

#
# The program's static data
#

.data

msg:    .string "Hello, world!n"
        length = . - msg

#
# Special section needed by the linker due to the C calling
# conventions in this platform.
#

.section ".opd", "aw"               # aw = allocatable/writable

.global _start
_start:
        .quad ._start, .TOC.@tocbase, 0

#
# The program's code
#

.text

._start:
        li      0, 4                # write(2)
        li      3, 1                # stdout file descriptor
        lis     4, msg@highest      # load 64-bit buffer address
        ori     4, 4, msg@higher
        rldicr  4, 4, 32, 31
        oris    4, 4, msg@h
        ori     4, 4, msg@l
        li      5, length           # buffer length
        sc

        li      0, 1                # _exit(2)
        li      3, 0                # return success
        sc

$ as -a64 -o hello.o hello.s
$ ld -melf64ppc -o hello hello.o

November 25, 2007 · Tags: cell, linux, powerpc
Continue reading (about 2 minutes)

Mad at the Cell SDK

I've been installing the Cell SDK 3.0 on two Fedora 8 systems at home — a PlayStation 3 and an old AMD box — and I cannot understand how someone (IBM and BSC) can publish such an utterly broken piece of crap and be proud of it. Sorry, had to say it. (If you are one of those who wrote the installer, please excuse me, but that's what I really think. Take this as a constructive criticism.)

Before saying that Fedora 8 is not supported and that I should only run this on Fedora 7, shut up. I am sure all the problems are there too, because none of them can be related to the system version.

Strictly speaking, it is not that the installer does not work, because if you follow the instructions it does. But it is a very strange program that leaves garbage all around your system, produces warning messages during execution and the garbage left around will keep producing warnings indefinitely. Plus, to make things worse, the network connection to the BSC — where the free software packages are downloaded from by yum — is extremely unreliable from outside the university's direct connection (that is, from home), which means that you will have to retry the installation lots of times until you are able to download all the huge packages. (In fact, that's what annoys me most.) And this is not a problem that happened today only; it also bit me half a year ago when installing the 2.1 version.

Let's talk about the installer, that marvelous application.

Starting with version 3, the SDK is composed of a RPM package called cell-install and two ISO images (Devel and Extras). When I saw that, I was pretty happy because I thought that, with the RPM package alone, I'd be able to do all the installation without having to deal with ISO images. It turns out that that is not true, as some components only seem to be available from within them (most likely the non-free ones, but I haven't paid attention).

Ah, you want to know what the SDK contains. Basically, it is composed of a free GCC-based toolchain for both the PPU and SPUs, the free run-time environment (the libspe2), a proprietary toolchain, the proprietary IBM SystemSim for Cell simulator and some other tools (a mixture of free and non-free ones). So, as you can see, we have some free components and some proprietary ones. You can, in fact, develop for the Cell architecture by using the free components alone. So why on earth do you need the proprietary ones? Why can't you skip them? Why aren't they available in some nice repository that I can use without any external "installer" and avoid such crap? That's something I don't get. (Maybe it's possible with some extra effort, but not what the instructions tell you.)

OK, back to the installer. So you need to copy the two ISOs in a temporary directory, say /tmp/iso and then run the installer by doing something like:

# cd /opt/cell
# ./cellsdk --iso /tmp/dir install

November 18, 2007 · Tags: cell, fedora
Continue reading (about 5 minutes)

PFC report almost ready

The deadline for my PFC (the project that will conclude my computer science degree) is approaching. I have to hand out the final report next week and present the project on July 6th. Its title is "Efficient resource management in heterogeneous multiprocessor systems" and its basic goal is to inspect the poor management of such machines in current operating systems and how this situation could be improved in the future.

Our specific case study has been the Cell processor, the PlayStation 3 and Linux, as these form a clear example of an heterogeneous multiprocessor system that may become widespread due to its relatively cheap price and the attractive features (gaming, multimedia playback, etc.) it provides to a "home user".

Most of the project has been an analysis of the current state of the art and the proposal of ideas at an abstract level. Due to timing constraints and the complexity of the subject (should I also mention bad planning?), I have been unable to implement most of them even though I wanted to do so at the very beginning. The code I've done is so crappy that I won't be sharing it anytime soon, but if there is interest I might clean it up (I mean, rewrite it from the ground up) and publish it to a wider audience.

Anyway, to the real point of this post. I've published an almost definitive copy of the final report so that you can take a look at it if you want to. I will certainly welcome any comments you have, be it mentioning bugs, typos, wrong explanOctations or anything! Feel free to post them as comments here or to send me a mail, but do so before next Monday as that's the deadline for printing. Many thanks in advance if you take the time to do a quick review!

(And yes... this means I'll be completely free from now on to work on my SoC project, which is being delayed too much already...)

Edit (Oct 17th): Moved the report in the server; fixed the link here.

June 19, 2007 · Tags: cell, linux, pfc, ps3
Continue reading (about 2 minutes)

Building the libspe2 on the PS3

The Linux kernel, when built for a Cell-based platform, provides the spufs pseudo-file system that allows userland applications to interact with the Synergistic Processing Engines (SPEs). However, this interface is too low-level to be useful for application-level programs and hence another level of abstraction is provided over it through the libspe library.

There are two versions of the libspe:

• 1.x: Distributed as part of the Cell SDK 2.0, is the most widely used nowadays by applications designed to run on the Cell architecture.
• 2.x: A rewrite of the library that provides a better and cleaner interface — e.g. less black boxes —, but which is currently distributed for evaluation and testing purposes. Further development will happen on this version, so I needed to have it available.
  

March 14, 2007 · Tags: cell, pfc, ps3, yellowdog
Continue reading (about 3 minutes)

PFC subject chosen

A while ago, I was doubtful about the subject of my undergraduate thesis (or PFC as we call it). At first, I wanted to work on a regression testing framework for NetBSD. This is something I really want to see done and I'd work on it if I had enough free time now... Unfortunately, it didn't fit quite well my expectations for the PFC: it was a project not related at all with the current research subjects in my faculty, hence it was not appropriate enough to integrate into one of these work groups.

So, after inverstigating some of the projects proposed by these research groups, I've finally settled on one that revolves around heterogeneous multiprocessor systems such as the Cell Broadband Engine. The resulting code will be based on Linux as it is the main (only?) platform for Cell development, but the concepts should still be applicable to other systems. Who knows, maybe I'll end up trying to port NetBSD to a Cell machine — shouldn't be too hard if that G5 support is integrated ;-)

The preliminary title: Efficient resource management in heterogeneous multiprocessor systems. For more details, check out the Project proposal (still not concreted, as you can see).

January 27, 2007 · Tags: cell, pfc
Continue reading (about 1 minute)