Free Software JavaFX
I've been keeping a (rather loose) eye out on the availability of
a fully free-software JavaFX for a while. Whilst all
distributions now ship OpenJDK by default and that works fine for
anything not JavaFX, the OpenJFX stuff has still been work in
progress and hasn't generally been available.
Anyway I thought i'd try to compile it myself. Of course the
first thing I noticed was that
a preview build IS now
available but I decided to keep going anyway so I could build
a complete integrated JDK/JRE rather than having to manually link
it in at runtime.
For the first part of the problem, following
instructions was quite straightforward, at least on Ubuntu
16.04. I manually installed a couple of the specific build tools
required and a couple of extra -dev packages. I didn't bother
with building the bundled webkit for now. Even on this gutless
under-RAM'd machine it didn't even take terribly long. I used
the openjdk 10 for the
Then I looked into integration witht the jdk (later section in the
build instructions). I was too lazy to read most of the build
instructions so I just went with configure && make, although after
a few iterations I settled on:
$ mkdir build
$ cd build
$ ../configure --with-import-modules=../../rt/build/modular-sdk \
$ make product-images
Some time later it's all done and I ran some simple tests and
Bob's your uncle. Stripping the debug symbols just reduces the
size of the install (significantly) although it's probably worth
keeping them at this early stage.
The slowest part was checking out the mecurial repositories.
Oh, I couldn't get the jdk tests to run :-/
just complained that it couldn't determine the JVM version.
Trying to search the interwebs has so far been fruitless - it's
not terribly important for now and I successfully ran
bootcycle-build which self-compiles after
bootstrapping which is at least some bit of testing.
Update: Not sure where I got it from but the jtreg I had
was out of date so it couldn't handle the new version string.
Starting on the jtreg
page got me the latest (4.2b12) and now the testing is
Although this box doesn't have much disk it does have enough for a
few big blobs so I thought I'd look into distributing a build
well!). To that end I sussed out setting up a private tracker
and seeding torrents directly from this machine.
Anyway i've mostly worked it out but it isn't quite ready but i'll
get to it eventually.
With the short lifetime of Java 9 and 10 i've basically just
ignored them completely - i'm still using 8.x everywhere. But
Java 11 is going to be LTS release so it's probably time to start
Having a fully free build of the whole platform is an important
part of that for my hobby code. Now I just need the motivation ...
Update 2: And NetBeans, which is still not supporting a Java
that was EOL'd 3 months ago. I'm sure they've nearly go the
licensing sorted out though!
Ok so I guess i'm a bit out of the loop (and search engines failed
me again, it took me hours to come across this stuff, mostly by
is going to removed from the JDK. And additionally the JRE
will also go away(?) and basically replaced with standalone
platform-specific builds via jlink. Actually i'm not sure how the
latter will work, maybe that's just wrt the openjdk; and in any
event Oracle are committing to another few years of Java 8 releases.
Given that all the 'action' is server-side I guess it makes some
sense. Just not for me!
Huh, I wonder why the current jdk build process allows linking in
the JavaFX module then. Unclear. Just doing so breaks the tests
so it's probably just the build system lagging slightly. Then
again the make_runargs.sh script used to run against the javafx
module doesn't seem to work against openjdk11 anyway. Well unless
there's some other meachanism but neither javafxpackager nor
javapackager get built - does jlink do all that now? So far my
searching has been pretty futile and a lot of the documentation is
either out of date or hard to find. I suppose it's still
basically work in progress and/or i'm just not that motivated to
dig deep enough.
Well I did waste the day anyway so whatever. At least AMD
announced some nice hardware and Intel made an arse of themselves
with their 'first' 5Ghz cpu demo which needed to be plugged into a
fridge to run for 10 seconds.
Oh, there is also openjfx in ubuntu repos, but it only goes to 8.
I can't remember if i looked at it in the past, probably did but I
think I was looking at java9 at the time not realising how short
lived it would be. I should've just stayed with 8.
Well I did find something about running against the standalone
this blog post. I guess all that javapackager stuff is gone
or something since webstart and applets are defunct? Not that I
ever used it anyway. Shrug.
and making a partial copy of the jre for every application will
be `easier' than just sending out a jar and having a central single
installation of a jre. Wot?
Everyone seems to be a bit upset that Oracle are only going to
provide commercial support for the oracle jdk11+ too - but given
how much they've invested in making the openjdk feature parity I
think they did ok for such an `evil' company. I mean, it's not
like they forked some public commons like Linux or something as
jjmpeg, jni, javafx
So I guess the mood took me, I somehow ended poking away until the very late morning hours (4am) the last couple of nights hacking on jjmpeg. Just one more small problem to solve ... that never ended. Today I should've been working but i've given up and will write it off, it's nearly xmas break anyway so there's no rush, and i'm ahead of the curve anyway.
I got this ported over and playing video fairly easily, and then went through on a cleanup spree. I removed all the BufferedImage, multi-buffering, and scaling stuff and a few other experiments which never worked. Some api changes allowed me to consolidate more code into a base class, and some changes to AVStream necessitated a different approach to initialising the AVCodecContext (using AVCodecParameters). I made a few other little tweaks on the way.
The reason I removed the BufferedImage code is because I didn't want to pollute it with "platform specific" code. i.e. swing, javafx, etc. I've moved that functionality into a separate namespace (module?).
My first cut just took the BufferedImage code and put it into another class which provides the functionality by taking the current AVFrame from the JJMediaReader video stream. This'll probably do but when working on similar functionality for JavaFX I took a completely different approach - implementing a native PixelReader() so that the native code can decide the best way to write to the buffer. This is perhaps a little more work but is a lot cleaner to use.
jjmpeg1 lets you scale images 'directly' to/from primitive arrays or direct ByteBuffers in addition to AVFrame. Since they have no structure description (size, format), this either has to be passed in to the functions (messy) or stored in the object (also messy). jjmpeg1 used the latter option and for now I simply haven't implemented them.
The PixelReader mentioned above does implement it internally but for code re-use it might make sense to implement them with the structure information as explicit parameters, and use higher level objects such as PixelReader/Writer to track such information. On the other hand the native code has access to more information so it also makes sense to leave it there.
I went a bit further and created a re-usable super-class that does most of the work and toolkit specific routines only have to tweak the invocation. This approach hides libswscale behind another api. The slice conversions don't work properly but they're not necessary.
So far I had public constructors and `finalisers' because otherwise the reflection code failed. That's a bit too ugly (and `dangerous') so I made them private. The reflection code just had to look up the methods and set them Accessible.
Constructor cc = jtype.getDeclaredConstructor(Long.TYPE);
Whilst working on JJMediaReader I hit a snag with the issue of ownership. In most cases objects are either created anew and released (or gc'd) by the Java code, or are simply references to data managed elsewhere. I was addressing the latter problem by simply having an empty release() method for the instance, but that isn't flexible enough because some objects are created or referenced the the context determines which.
So I expanded the Java-side object tracking to include a `refer' method in addition to the 'resolve' method. `resolve' either creates a new instance or returns and existing one with a weak-reference object which will invoke the static release method when it gets finalised. `refer' on the other hand does the same thing but uses a different weak-reference object which does nothing.
I then noticed (the rather obvious) that if an object is created, it can't possibly 'go away' from the object tracking if it is still alive; therefore the `resolve' method was doing redundant work. So I created another `create' method which assumes the object is always a new one and simply adds it to the table. It can also do some checking but i'm pretty sure it can't fail ...
If on the other hand the underlying data was reference counted then the `resolve' method would be useful since it would be possible to lookup an existing object despite it being `released'. So i'll keep it in CObject.
As part of this change I also improved CObject in other ways.
I was storing the weak reference to the object itself inside the object so I could implement explicit release and to avoid copying the pointer. I removed that reference and only store the pointer now. The WeakReference it already tracked in a hash table so I just look it up if I need it. This lets me change the jni code to use a field lookup rather than a function call to retrieve it (I doubt it makes much perf difference but I will profile it at some point).
I also had some pretty messy "cross-layer" use of static variables and messy synchronisation code. I moved all map references to outside of the weak reference routine and use a synchronised map for the pointer to object table.
For explicit release I simply call .clear() and .enqueue() on the WeakReference - which seems to do the right thing, and simplifies the release code (at least conceptually) since it always runs on the same thread.
I tried a few more variants on the OpenCL rendering code but none are all that fast - the overheads kill it and whilst it does free the CPU up a little bit it isn't much. Probably not worth more time unless I look at OpenGL instead.
I added resizing. It meant I had to add some locking to the snapshot routine but it only needs to lock around the resize operation so it adds almost no overhead to normal operation (merely detecting a resize has occurred). I'm not yet sure what's supposed to happen with saved cursors and alternate screens when a resize occurs, probably just clip to size. Unfortunately there seems to be no way to set the WM_NORMAL_HINTS on the javafx stage, so there's no way to make it size to cells properly.
I did a bunch of benchmarking and profiling. One thing I tried was another test to compare to xterm - now at full-screen. Running "find . -name '*.c' | xargs cat" from the root of the linux 3.19.8 source tree. After a couple of runs this is about 25 seconds on termz, and 16 minutes in xterm. Well. Yeah it's a silly test but all those ls -l's during the day add up.
Looking at the memory profiler it doesn't really use too much heap during operation, just a few MB and most of that is the image and javfx. I mean nor should it, there isn't data structures to maintain. But having multiple compilers in ram (jvm, OpenCL), their generated output, and all the added overhead of the runtime support needed for those really adds up so it's very very fat in practice. I guess if multiple terms ran on the same jvm it would be ok.
So at this point i'm not really sure what i'll do with it. I'll probably poke at the edges when i'm bored and eventually when I get around to it I will dump what I have to my software site as the result of a weekend-and-a-bit-hack.
After that I'm not sure. It's actually quite functional and robust already (well, compared to the effort in) and wouldn't take much more work to turn it into a usable terminal for me - add some scrollback (pretty simple), mouse selection stuff (not that hard), and sort out some of the keyboard details (reading obtuse documents and testing). So maybe that will happen.
If I got that far adding the "10x20" typeface would probably be on the cards. Fixed-size outline fonts would be possible by just pre-rendering them but to me they just aren't terminal fonts.
Anything further such as a re-usable term component (which might actually be of use to the world) would require substantially more work on the i18n side of things and I don't feel like learning enough to do that properly.
I did a bit poking at the java makefile stuff and it's to the point where i'm using it for out-of-ide builds of termz and zcl and will look into using it on other projects. That's the best way to find bugs/what works and what doesn't and previous attempts never got that far. For all it's gnumakefile obtuseness it's really rather compact at under 200 lines excluding comments and I didn't put any effort into making it particularly small. And that includes targets for javadocs, source jars, and binary builds.
I was just going to "try something out" while I waited for the washing to finish ...
... so after a long and full day of hacking ...
The screenshot is from an OpenCL renderer. Each work item processes one output pixel and adds any attributes on the fly, somewhat similar in effect to how hardcoded hardware might have done it. I implemented a 'fancy' underline that leaves a space around descenders. The font is a 16x16 glyph texture of iso-8859-1 characters. I haven't implemented colour but there's room for 16.
On this kaveri machine with only one DIMM (== miserable memory bandwidth) the OpenCL routine renders this buffer in about 35-40uS. This doesn't sound too bad but it takes 3uS to "upload" the cell table input, and 60uS to "download" the raster output (and this is an indexed-mode 8-bit rather than RGBA which is ~2x slower again), but somehow by the time it's all gone through OpenCL that's grown to 300-500uS from first enqueue to final dequeue. Then add on writing to JavaFX (which converts it to BGRA) and it ends up ~1200uS.
I'm using some synchronous transfers and just using buffer read/write so there could be some improvements but the vast majority of the overheads are due to the toolkit.
So I guess that's "a bit crap" but it would still be "fast enough". For comparison a basic java renderer that only implements inverse is about 1.5x slower overall.
But for whatever reason the app still uses ~8% cpu even when not doing anything; and that definitely isn't ok at all. I couldn't identify the cause. Another toolkit looks like a necessity if it ever went beyond play-thing-toy.
I got bored doing the escape codes around "^[ [ ? Ps p" so it's broken aplenty beyond the bits I simply implemented incorrectly. But it's only a couple days' poking and just 1K3LOC. While there is ample documentation on the codes some of the important detail is lacking and since i'm not looking at any other implementation (not even zvt) i have to try/test/compare bits to xterm and/or remember the fiddly bits from 15 years ago (like the way the cursor wrapping is handled). I also have most of the slave process setup sorted beyond just the pty i/o - session leaders, controlling terminals, signal masks and signal actions, the environment. It might not be correct but I think all the scaffolding is now in place (albeit only for Linux).
FWIW a test i've been using is "time find ~/src" to measure elapsed time on my system - after a couple of runs to load everything into the buffer cache this is a consistent test with a lot of spew. If I run it in an xterm of the same size this takes ~25s to execute and grinds big parts of the desktop to a near halt while it's active. It really is abysmal behaviour given the modern hardware it's on (however "underpowered" it's supposed to be; and it's considerably worse on a much much faster machine). The same test in 'termz' takes about 4.5s and you'd barely know it was running. Adding a scrollback buffer would increase this (well probably, and not by much) however this goes through a fairly complete UTF-8 code-path otherwise.
The renderer has no effect on the runtime as it is polled on another thread (in this instance via the animation pulse on javafx). I don't use locks but rely on 'eventual consistency'. Some details must be taken atomically as part of the snapshot and these are handled appropriately.
Right now I feel like i've had my fill for now with this. I'm kinda interested, but i'm not sure if i'm interested enough to finish it sufficiently to use it - like pretty much all my hacking hacked up hacks. Time will be the teller.
Every now and then I think about the sad state of affairs regarding terminal emulators for X11. It's been a bit of a thing for a while - it's how i ended up working at Ximian.
I stopped using gnome-terminal when i stopped working on it and went back to xterm. I never liked rxvt or their ilk and all of the 'desktop environment' terminal emulators are pretty naff for whatever reason.
xterm works and is reliable but with recent (being last 10 years) X Windows System servers the text rendering performance plummeted and even installing the only usable typefaces (misc-fixed 6x13, and 10x20, and sometimes xterm itself) became a manual job. Whilst performance isn't bad on this kaveri box I also use an uber-intel machine with a HD7970 where both emacs and xterm runs like an absolute pig whenever any GL applications are running, and it isn't even very fast otherwise (i'm talking whole desktop grinding to a halt as it redraws exposes at about 1 character column per SECOND). It's an "older" distribution so that may have something to do with it but there is no direct indication why it's so horrible (well apart from the AMD driver but i have no choice for that since it's used for OpenCL dev). I might upgrade it to slackware next year.
Anyway I started poking last night at a basic xterm knockoff and got to the point of less sort of running inside it and now i'm thinking about ways I might be able to implement something a bit more complete. I'm working in Java and have a tiny bit of JNI to get the process going and handle some ioctl stuff (which seems somewhat easier now than it was in zvt, but portability is not on the agenda here).
TermZ? Glyphs are greymaps extracted directly from the PCF font.
When I wrote ZVT the primary goal was performance and to that end considerable effort was expended on making a terminal state machine which implemented zero-copy and zero-garbage algorithms. zero-copy is always a good thing but the zero-garbage was driven by the very slow malloc on Solaris at the time and my experience with Amiga memory management.
Another part of the puzzle was display and the main mechanism was inspired by some Amiga terminal emulators that used COPPER lists to re-render rows to the screen in arbitrary order without requiring them to be re-ordered in memory (memory bandwidth was a massive bottleneck when using pre 1985-spec hardware in 199x). I used a cyclic double-linked (exec) list of rows and to implement a scroll I just moved a row from the start to the end of the list which takes 8 pointer updates and a memset to clear it (and it also works for partial screen scrolls). By tracking the last row a given one was actually displayed at I could -at-any-point-later- attempt to create an optimal screen-update sequence including using blits for scrolling and minimising glyph redraws to only those that had changed. The algorithm for this was cheap and reliable if a little fiddly to get correct.
This last point is important as it allows the state machine to outpace the screen refresh rate which always becomes the largest bottleneck for terminal emulators in 'toolkit' environments. This is where it got all it's performance from.
new hardware, new approach
Thinking about the problem with current hardware my initial ideas are a little bit different.
I still quite like the linked list storage for the state machine and may go back to it but my current idea is instead to store a full cell-grid for the displayable area. I can still make full-screen scrolling just as cheap using a simple cyclic row trick (infact, even cheaper) but sub-region scrolling would require memory copies - but at the resolution of 4-bytes-per-glyph these are insanely cheap nowadays.
This is the most complex part of the emulator since it needs to implement all the control codes and whatnot - but for the most part thats just a mechanical process of implementing enough of them to have something functional.
I would also approach rendering from an entirely different angle. Rather than go smart i'm going wide and brute-forcing a simpler problem. At any given time - which can be throttled based on arbitrary metrics - I can take a snapshot of the current emulator screen and then asynchronously convert that to a screen display while the emulator continues to run on it's own thread.
For a basic CPU renderer it may still require some update optimisation but given it will just be trivial cell fonts to copy it probably wont be appreciably cheaper to scroll compared to just pasting new characters every time. And obviously this is utterly trivial code to implement.
The ultimate goal (and why the fixed-array grid backing is desirable) would be to use OpenCL or OpenGL (or more likely Vulkan if it ever gets here) to implement the rendering as a single pass operation which touches each output pixel only once. This would just take the raw cell-sized rectangle of the terminal state machine as it's only variable input and produce a fully rendered and styled framebuffer as the result. Basically just render the cells as a low-res nearest-neighbour texture lookup into a texture holding the glyphs. The former is a tiny tiny texture in GPU terms and rendering a single full-screen NN textured quad is absolutely nothing for any GPU. And compared to the gunk that is required to render a full-screen of arbitrary text through any gui toolkit ever it's many orders of magnitude less effort.
Ideally this would only ever exist at full-resolution in the on-screen framebuffer memory which would also make it extremely cheap memory wise.
But at least initially I would be going through JavaFX so it will instead have to have multiple copies and so on. The reason to use JavaFX is for all the auxiliary but absolutely necessary fluff like clipboard and dnd operations. I don't really like tabbed terminals (I mean I want to use windows as windows to multitask, not as a stack to task switch) but that is one way to ameliorate the memory use multiplication this would otherwise create.
So to begin with it would be extremely fat but that's just an implementation detail and not a limitation of the design.
Still mulling that over. It's still a lot of work even if conceptually it's almost trivial.
Rather than 'play some more' i went head-first into writing a better display/model/control thing for the synthesiser stuff.
Doesn't look much different on the surface since the stylesheet is much the same but pretty much everything is backed by controls and skins and observable properties.
I have a 'Workbench' control which is the main controller and it handles the human wiring process and so on. I abstracted the connection points as 'Sockets' in a way which allows me to add other non-JSyn objects such as the dials which just feed out their value to some input port. Not shown but you can also reconfigure input ports to change their name and bounds so that they connect properly to dials.
I added the ability to unplug and move the ends of a given wire which works quite nicely. If you grab a wire near one end and 'pull' it far enough it detaches and becomes a free end, which can either be attached to some other compatible socket or letting go just deletes that wire.
I/O next I guess? Always the pain ...
FWIW The circuit above makes a pretty basic "idling chopper" "fop-fop" sound.
Connecting the dots (with wires)
Yesterday I hacked up a "workbench" gui for JSyn. I just started experimenting with auto-generating the gui based on the UnitGenerator interfaces, it's designed with this in mind so it is fairly simple.
Blocks are added by clicking on the workspace, wires are connected by dragging a line between the discs, blocks can be arbitrarily dragged around, wires or blocks are deleted with delete or backspace keys; typical basic graphical user interface stuff.
One of the more difficult problems was how to wire up the ports ...
The difficulty arises because one uses container classes to form grouped gui components and do the layout, and then you want to be able to bind a linkage between an inner component of the group and components working in a different local coordinate system.
A diagram of the layout hierarchy as it currently exists helps to explain (this is just a prototype so the layout is a bit shit).
- ----+---- -
- ----+---- -
Generator (BorderPane) Wire (Line)
So the Line representing the wire needs to connect to the Ellipse level, but the Line is in the coordinate space of the lower Pane and the Ellipse is in the coordinate space of the HBox which is in the coordinate space of the EndPoint, which is ...
So I created a new set of properties socketX, socketY, which reside on each EndPoint which maintain the location of the centre of the Ellipse relative to the lower Pane. The calculation is straightforward and based on transforming to and from scene-relative coordinates. It gets updated whenever the EndPoint is moved (relative to the Generator, i.e. after layout), or when the Generator is moved (i.e. relative to the Pane). The Wire can then just bind to these locations using javafx properties. I've got each generator maintaining the endpoint updates relative to it's parent but i'll probably have to come up with something better.
There's still a problem to solve though; the user interface allows you to plug a wire in to one socket and then drag it to another socket (i.e. click and drag). Unfortunately javafx currently has no way of picking a widget - i.e. looking it up by location or by the mechanism the event system uses. At first I tried using drag and drop mechanisms but it interfered with other gui operations and I couldn't find a way to show the wire following the mouse pointer.
So unfortunately I have to maintain a separate list of all the EndPoints in a model which I then scan for a match during the drag gesture. But given that this allows me to use layouts and stylesheets and so on, this is acceptable.
I created a new square wave oscillator type for JSyn as i was surprised the one it comes with doesn't have a duty cycle (pulse with) parameter. It was one of the more useful features of the square wave oscillator on the SID chip and features heavily in C=64 music. I think it could be created using a sawtooth generator and a gate but this is easier to use and was simple to make.
The circuit above generates a rather grating undulating / phasing sound effect like a 'robot thinking', or with a lower frequency a (metallic) misfiring engine.
I guess i'll add more of the unitgenerators from JSyn and do some experimentation with the code I have. If it's interesting enough i will probably dig deeper into making it a real application; it will need a better system model and probably a way to create a control panel separate from the inner workings of a circuit, as well as oscilloscope and spectrum analyser blocks (JSyn has most of this but it's all Swing).
Been getting pretty bored of being stuck in the house. Enough so that I coded up a basic space invaders game for something to do. At this point almost everything is there apart from the mother ship going past and some other game state things or animations. I've tried to keep the ship movement and wave structure correct (something clones usually try to 'improve') based on the C=64 version.
I'm using javafx for the graphics (i.e. most sprites are just a Rectangle - the actual shape isn't terribly important for the mechanics, and saves copyright issues) but the barriers are WritableImage's and i do the hit detection and bomb erosion on separate image arrays that back them. Because of the way the hit detection works it would probably be easier just to do everything on a bitmap manually but I didn't feel like starting from scratch after starting with a scenegraph version.
Yesterday I added some sound effects. The invader explosion sound is a flat beep but the rest are more or less correct including the 4-note 'tune' which speeds up as the invader movement does. It doesn't sound right when there's only one left but otherwise it's close.
This lead me to play a bit with the java sound api ... which is somewhat better than I thought it was; e.g. it provides automatic mixing and so on. It also meant I had to do some sound synthesis for the tune and effects. So far i'm generating sample Clip objects from some synthesis code I wrote myself but today I looked into it deeper and found some existing synthesiser libraries that should let me generate some better sounds. But i'll need to play with them a fair bit before getting anywhere. Actually unless I find something i'll probably have to write some tools for it anyway which starts to make it a fairly significant effort.
Last time i played with synthesisers was with the SID around 1990 and I never got very far with that, so i'm starting from a point of somewhat ignorance here.
Copyright (C) 2018 Michael Zucchi, All Rights Reserved.Powered by gcc & me!