Shared Libraries n Stuff
Hmm, another long weekend followed by what turned into a long week. Damn I'm tired.
Well I gave up on building XBMC on the beagleboard for now, and I haven't had time to work on setting up a cross compilation environment yet either. Work is getting in the way too much, just too worn out to concentrate enough. I should try this weekend, although I need to get outside a bit too for some exercise and my own sanity.
I spent the rest of the long weekend past trying to grok shared libraries and thinking about how to get newlib into one for WoofƆs. I didn't make a lot of progress. It would be nice just to use something like Amiga libraries which are more like objects, including an instance pointer to all functions, but that just makes it too hard to re-use existing code. Partly because all the code also must be re-entrant and not reference .data or.bss, not to mention the object pointer. Easier to map/load, not so easy to port existing libraries. I just about had a handle on the whole elf-shared-library thing, but I was a bit too tired/distracted so it's still a bit up in the air.
Well at least one nice thing is it only takes 17 seconds(!) to do a complete build of newlib on my workstation, althoughI fear I will need a few iterations before I work it out. I was trying to enable _REENT_ONLY, but it seems to be an old option which is no longer supported and fixing it all up will be more hassle than it's worth, so I will probably just have to work out elf shared libraries to make any progress there.
It's not that I really need a C library just yet, but I was thinking about how to write the application server and how to hook system startup into application startup, i.e. accessing dynamic memory and so on, and so I was just thinking of killing two birds with one stone. Might have to leave that on the back burner for a bit, and try something a bit more modest. Actually I should probably put it all on the backburner for a wile, I have other things that need attention.
I didn't do anything on the yard at all last weekend - a bit hungover Saturday and then the weather turned. It's been a bit cold and crappy most of the week (in relative terms) which is always pretty draining. And the days are so short already ... (particularly when you get up at 11 or 12!)
Sleep. Damn. Eating poorly, sleeping worse. It adds up. My mouth has been a bit sore so I have avoided using the mouth splint every night - but boy do I notice if I don't use it. It's like the `Rain God' truck driver in So Long and Thanks for All the Fish who upon experiencing rain so heavy he imagines it makes no difference whether the wipers are on or off tries turning them off, only to discover it really does make it a lot worse. However to his dismay it doesn't get any better when he turns them on again. That's exactly what it is like with the mouth splint and sleep apnoea.
Well i'm glad I stayed in rather than going to the pub tonight - starting to feel a bit weird, and last weekend was too depressing - my drinking buddy went off to a show and I had a couple more at another pub, and what I can remember got pretty boring by the end.
Well I integrated a timer interrupt into WoofƆs and got that to work eventually - the MMU makes things like this very fun as I have to map everything I need to use. A few silly bugs in the MMU code didn't help either.
After a 7 hour stint last night I finally booted up an executed a 'user process', including sending it the initial welcome message. It doesn't do much other than call a debug printf function to say hello, but given that it is running in it's own address space successfully that seems a reasonable achievement.
Interrupts are going to be a bit of a pain. Because I want user code to deal with interrupts the kernel has to clear the hardware interrupt status bits itself - which means it needs to know about all the devices that might produce interrupts and how to turn them off. I can't see any way around this right now. I also need some way of letting the user-code know what triggered the interrupt, which is another issue - even with 96 interrupts there is some double-up on the interrupt source. I guess there's always the mailbox idea, or a syscall to query it. There might be some hardware where the interrupt needs processing immediately too, but i'll come to that when I find it.
Hmm, next thing might be to think about devices and get a serial device working.
Finally got timers `worked out' - i'd played with it before but couldn't get it to work - but it turned out that the code was basically fine, but I had a bug in the irq `enable' routine ... der. They are really rather simple devices that work much the same in almost all microprocessors ever made ..., so it was a bit embarrassing it took this long.
It's using the 32 768Hz clock - which means you have to use some extra hardware logic to get a true 1Khz divisor, but the manual explains that pretty clearly as referenced in the source.
Source is in irq-timer.c.
I'm at the point in WoofƆs that I need at least one timer to get much further so it seemed a good time to poke it into puppybits. I managed to squeeze in a little time to play with WoofƆs over the last few nights and got the code that i'd written a few days ago to load and run - after a stack of mucking about as usual. It doesn't do much - with the trivial round-robin scheduler and no interrupts once the memserver task goes to sleep the idle task starts and is never pre-empted :). I had some linking issues, but the main problem was that for some silly reason I was calling some assembly from C as a function *pointer* not a function and it took me far too long to spot it.
But there's an excuse this time. It's been a really long week - 49 hours of work, quite a few very late nights, an on-site meeting I had to wake up at 7am for, a couple of midnight meetings for GSOC - one of which I missed because I fell asleep, and then I woke up at 2am and didn't crash again till after 8am this morning. Not sure 'tired' covers it. The sunrise wasn't bad though.
That's it for summer I guess.
Daylight savings just finished here - and it's already throwing me out. After plugging away at the computer all afternoon I thought it must've been 10 or something as it seemed to have been dark for so long, but it was only 7:30 ... Good time to call it a day I guess.
Yesterday I got a whole lot of work out of the way on the retaining wall, I should've done more today but I was too lazy. The weather was too nice too - although then I just spent most of the day inside instead of taking advantage of it. Once I sit down to hack it's pretty well game over ... Probably should've made the effort though - looks like weather is turning wet soon, and I need to get some of the stuff done whilst it's still try. Maybe an hour or two over the next two days might do it.
Well that's Easter done, back to work tomorrow.
Was up pretty late hacking away on some WoofƆs stuff, and then continued it for most of today. For the most part it's just re-arranging stuff I already have from puppybits, or my previous x86 hacking. Pretty time consuming though, trying to tie it all together. It also needed quite a bit of re-thinking and re-jigging along the way.
I've got it all building, and it currently launches two tasks, one which immediately goes to sleep waiting for work to do, and the idle task which blinks the led. But that isn't really any more than I already have in puppybits, so I need to start testing the other stuff - per-process virtual memory, message passing, and so on. These things are a bit hard to play with in isolation since so much support is needed first. Hmm, I really need to get timers and timer interrupts working too, but I can test that in isolation.
The architecture which i've thought about so far does seem to be holding together at least - no big surprises have come along ... yet. Although that assumes the stuff i've written actually works too. So far I have a 'memserver' which is tightly bound to the kernel - it is basically a kernel thread since it accesses the kernel memory directly. It is used to create all in-kernel objects - directly adding them to the kernel in some cases, or where necessary through lightweight system calls to register the new resource. This lets me avoid any dynamic memory in the kernel itself, and by using the right data structures or the occasional lightweight system call I don't have to worry about serialisation either (well, once i've got it right).
I had been thinking about all sort of exotic data structures like trees or hash tables to locate resources based on an object id - but these all have various issues. Execution time, serialisation, and so on (I have implementations that use no dynamic memory, so that wasn't an issue). So in the end I settled for a simple array for many of the objects - it probably uses less memory anyway, and certainly needs less code. It also allows me to update and access it atomically from multiple threads of execution without trouble.
The whole 'kernel' is only 10 system calls so far - and although I still need a few more for interrupts, it shouldn't be many ... I think. Of course most of the work is in the servers, and the kernel is just passing data around. Even in those 10 I already have some 'helper' syscalls too - they aren't strictly necessary but combine a couple of system calls into one, there is probably scope for some more of those.
I had a little chicken and egg problem with message allocating - you need to send a message to the memory server to allocate new messages before they can be sent via the kernel. I think I have worked out a solution - part of the process start-up will be to send a system message to the new process. This message can then be used to ask for more, or just as a general purpose message container. Still, I might need some other more direct mechanism since message ownership passes every time they are forwarded, and it is possible for them to get `lost'.
I might have to look into a mailbox mechanism I guess, although I don't want to have too many options for IPC. After I'd written the basic message allocation system I thought it would be a bit bulky to do too often - well the point is not to in the first place, but it seemed a bit heavy. I looked into a simpler mechanism of message passing using limited registered arguments. But it just didn't seem that useful - because you can only pass primitive non-pointer types. As soon as you want to pass buffers around the mechanism falls down, and you need even more complex support code (like 'far' copies), which then needs to perform extra security checks, and so on. One `freebie' of the message passing mechanism as i've envisioned it so far is it sort of self-checks. Nothing the kernel deals with needs checking in detail because any addresses are from trusted sources or have already been verified, or just integer representations of virtual addresses it never looks at. And servers can perform fairly cheap checking since all data must fit in a fixed bound.
Might sit on all that for a while now.
Finally cleaned up and checked in the kernel start code, and a simple demo which uses it.
I still need to do some prototyping of various low-level components before I can make a whole, but I might kick off the part of the whole I can do already because i'm getting a bit bored of just working on these small fragments.
Keeping with the canine theme, i'm going to call it ...
Or maybe WʊfƆs, although I think the larger one is more aesthetically pleasing (the letters are based on the international phonetic alphabet if you were wondering).
I'll probably just keep it in the puppybits project, although keep the code separate. With work and home being as they are I haven't had much time nor energy to work on this stuff, so things aren't going to progress at any great rate. Speaking of which I think it's time to hit the yard and get dirty shoveling. Two days of Easter down and all i've managed so far is ~97km of cycling, a hangover, and an improved tan.
I've been umming and aahing about working on a ps3 version of the kernel, because although there is plenty of other stuff to do, i've gotten to the point i'd like to have a framebuffer to work in. And since VGA hardware is such a pita to work with ... maybe the ps3 is the go.
So I did a lot of reading and digging, about powerpc64 hardware (hmm, this is really a mainframe chip, not a game console one!), hypervisor calls (so little documentation), instruction sets, etc etc. Hmm, a lot of work, but it seemed like a good challenge.
But then I thought i'd look at extracting a vga driver from some library - i looked at svgalib only because it had the least dependencies. I thought maybe I could get something going just to get started, or at least evaluate the size of the task. And after wasting a couple of days and very late nights on this ... I finally (re)discovered bochs and qemu had their own framebuffer device which was trivial to setup (I had seen the page a couple of weeks ago and 'noting it for later' promptly forgot it). Ho hum, what a total waste of time that was then. I still need a real driver if i want to get something up on real hardware, but it isn't exactly a priority right now.
So given I now have a framebuffer to work with I might postpone the ps3 stuff. I'm still mulling over an ARM based stuff too, so maybe I will look at that next instead - i imagine it will be somewhat simpler, and I can use qemu for that too.
Now I have a framebuffer ... what to do with it. OpenVG looks interesting, I might start there.
Cells and ratty rodents
Got off me arse and posted a new intro to Cell tutorial. It gets into SIMD coding so it's one of the more interesting ones.
And I managed to get a ps2 mouse driver working, of sorts (well, i'm getting the bytes from the port). And what a pile of shit the PS2 AUX port is. There doesn't seem to be any official documentation, just a few old text files from the days before the internets. Anyway, after a lot of mucking about I got it to work on bochs, qemu and and old PC I have - so that's good enough for me and it didn't take too much code. I'm still not sure if I should combine the keyboard and mouse 'devices' since they share the same io port.
Of course, in the process of testing on real hardware I found everything was broken. Everything. Ho hum. After much frobnication I found the APIC maybe wasn't as easy to use as I thought - or it's just buggy. And my old laptop doesn't even have one (early celeron). So I had to remove all the APIC code so it booted (i'm not doing any run-time stuff yet). And then I moved to using the RTC for timing instead. But that didn't work on real hardware either. Arg. In the end I got it to work but i'm not sure if it was just setting values directly (rather than read-twiddle-write) or clearing the interrupts first.
The apic thing is a bit of a bummer, I guess I'll need to use the pic instead for timing. Which sucks because it can only measure very short periods of time. I suppose if I use the RTC for longer periods - either by just using the 64Hz signal to count down, or the alarm function, and just resort to the pic for the last bit if more accuracy is required, it should be an ok balance between accuracy, overhead, and simplicity.
Hmm, wonder what to do next. A framebuffer would be nice - i'd really rather piss off the text mode entirely. And a disk driver - although then i'd need a filesystem too. Maybe it'd be less effort working out the hypervisor on the ps3 ... Hmm, perhaps a forth based monitor? Something to think about ...
Press Any Key
Well that was a strange weekend. I felt very odd Saturday - very sore throat, dizzy, really nasty headache and a bit of a fever for a while. The fever and dizziness went but I've still got a damn sore throat (a bit less so). I slept a lot.
I managed to get sucked in to poking on code again ... and once I start it's hard to stop.
I wrote up a basic keyboard device - it's a user-mode server which listens to an incoming message port for requests messages and for interrupts. If it gets an interrupt it buffers the keys (and provides decoding/etc), and if it gets a request it returns a key from the buffer (or queues it until one comes along). Very simple code. Then I wanted to have it synthesize key repeats too -- I find the pc keyboard repeats too slowly -- so that meant looking at writing a timing sub system. I tried to think of how these devices should be setup and initialised but for now I've just hardcoded their process creation from the kickstart routine, and use a simple manual process for accessing them via a public port name.
So then I had to work out how to use the APIC for timing - I'll leave the PIC timer for scheduling for now. And with that, I wrote a preliminary 'timer.device', although I haven't tried to see if it works yet. I ummed and ahhed over putting the timing service directly in the kernel, rather than in a user-service. But the kernel can only send signals to processes, and the timer device can send messages, so it seems like it's more useful, even if there is a somewhat more overhead involved. Sigh, I wish there was another APIC timer or two, it would be nice to have a good reliable period counter as well as one to be reprogrammed for varying intervals.
Copyright (C) 2018 Michael Zucchi, All Rights Reserved.Powered by gcc & me!