Michael Zucchi

 B.E. (Comp. Sys. Eng.)


android (44)
beagle (63)
biographical (82)
business (1)
code (56)
cooking (29)
dez (6)
dusk (30)
ffts (3)
forth (3)
free software (4)
games (32)
gloat (2)
globalisation (1)
gnu (4)
graphics (16)
gsoc (4)
hacking (414)
haiku (2)
horticulture (10)
house (23)
hsa (6)
humour (7)
imagez (28)
java (216)
java ee (3)
javafx (48)
jjmpeg (67)
junk (3)
kobo (15)
linux (3)
mediaz (27)
ml (15)
nativez (3)
opencl (117)
os (17)
parallella (97)
pdfz (8)
philosophy (26)
picfx (2)
politics (7)
ps3 (12)
puppybits (17)
rants (134)
readerz (8)
rez (1)
socles (36)
termz (3)
videoz (6)
wanki (3)
workshop (2)
zedzone (13)
Monday, 20 July 2015, 18:29

digest nonce

Intended to do nothing with the weekend ... but then i had "nothing better to do" so did a bit more hacking on the server. I had intended to go look for an updated distro for the xm, but promptly forgot all about it.

I did a bit of work on a `cvs-like' tool; to validate the revision system and to manage the data until I work out something better. The small amount I did on this exposed some bugs in some of the queries and let me experiment with some functions like history logging. The repository format is such that data updates are decoupled from metadata updates so for a history log they have to be interleaved together. I also came up with a solution for delete and other system flags: I already had an indexed 'keyword' set for each artifact so I just decided on using that together with single-character prefixes to classify them. Apart from these flags I will also use it for things like keywords, categories, cross-reference keys, and whatever else makes sense. System flags probably don't need indexing but it's probably not worth separating them out either. But the short of it is I can now mark a revision as deleted and it doesn't show up on a `checkout' of the branch containing that mark.

I did a bit of investigation into berkeley db je to see about some features I was interested in and along the way upgraded to the latest version (damn that thing doesn't sit still for long). This is also AGPL3 now which is nice - although it means I have to prepare a dist before I can switch anything on. Probably for now i'll stick with what I have; I was looking into having a writer process separate from the readers but I didn't get to reading very much about the HA setup before moving onto something else. It's just getting a bit ahead of where i'm at.

The driver of this is more thinking about about security than scalability. It's not really a priority as such; but its too easy to do stupid things with security and i'm trying to avoid any big mistakes.

So I had a look at what plain http can do and toward that vain implemented a chunk of RFC2617 digest authentication. This guy's code helped me get started so I could just skim-read the RFC to start with but eventually I had to dig a bit further into the details and came up with a more reusable and complete implementation. The main differences are requiring no external libraries by using javase stuff and the nonces are created randomly per-authentication and have a configurable timeout. It all works properly from a browser although nobody seems to use any http auth anymore; I presume it's all just done with cookies and if we're lucky some javascript now (and perhaps, or not, with ssl).

After I did all this I noticed the Authenticator class that can be plugged into the HttpContext and with not much work I embedded it into a DigestAuthenticator. Then made sure it will work free-threaded.

One problem with digest auth is that a hash of the password needs to be stored in plaintext. Although this means the password itself isn't exposed (since people often reuse them) this hashed value is itself used as the shared secret in the algorithm. And that means if this plaintext hash is accessed then this particular site is exposed (then again, if they can read it then it's already been completely exposed). Its something I can put in another process quite easily though.

I'm not sure if i'll even use it but at least I satisfied my curiosity and it's there if i want it.

Oh, along the way I (re)wrote some MIME header parsing stuff which I used here and will help with some stuff later. It's no camel but I don't need it to be.

On Sunday I thought i'd found a way to represent the revision database in a way that would simplify the queries ... but I was mistaken and since I still had nothing much better to do end up filling out some of the info implementation and html translator and found a way to approximately align the baselines of in-line maths expressions.

Tagged hacking, java, wanki.
Thursday, 16 July 2015, 21:41

bring out the maths.

I wasn't going to bother with supporting the @'math{}'[sic] command in my texinfo backend, but a quick search found jlatexmath so I had a bit of a poke and decided to drop it in.

He's some examples as rendered in my browser:

The first of each row is rendered using jlatex math using Java2D, and the second is rendered by the browser (svg). They both have anti-aliasing via alpha and both render properly if the background colour is changed. Most of the blur differences seems to be down to a different idea of the origin for pixel centres although the browser is probably hinting too (ugh). But the png's are also only 4 bit as well; but they hold up pretty well and actually both formats look pretty decent. Alas the poor baseline alignment, but this is only a quick hack and not a complete typesetting system.

The SVG should at least scale a bit; unfortunately it tends to get out of alignment if you scale too much. Hinting maybe fucking it up?

When I did it I first I hacked up a really small bit of code which directly outputs SVG. It implements enough of a skeleton of a Graphics2D to support the TeXIcon.paintIcon() function. Only a small amount is needed to track the transform and write a string or a rectangle.

As an example, @math{E=mc^2} gets translated into this:

<svg xmlns="http://www.w3.org/2000/svg" width="65" height="18" version="1.1">
<text x="2.00" y="15.82" font-family="jlm_cmmi10" font-size="16.00">E</text>
<text x="18.26" y="15.82" font-family="jlm_cmr10" font-size="16.00">=</text>
<text x="35.14" y="15.82" font-family="jlm_cmmi10" font-size="16.00">m</text>
<text x="49.19" y="15.82" font-family="jlm_cmmi10" font-size="16.00">c</text>
<text x="56.12" y="9.22" font-family="jlm_cmr10" font-size="11.20">2</text>

There are some odd limitations with svg used this way, no alt tag or way to copy the image pixels is a pretty big pair of problems. So I also looked into inline PNG and since I was going to that much effort seeing how small I could make it by using a 4-bit image.

After a bit of poking around I worked out how to generate a 4-bit PNG with the correct alpha directly out of javase. I render to a normal 8-bit image and then copy the pixels over to a special 4-bit indexed image using get/setRGB(), and the ImageIO PNG writer writes it correctly. Rendering directly to the image doesn't work (wrong colour selection or something to do with the alpha channel), nor does image.createGraphics().writeImage(8bitimage), although a manual data elements write should and will be the eventual solution.

It makes for a compact image and in base64 the image is about the same size as the svg.

<img alt="e=mc^2" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEE

FWIW this is how I create the image that I write to the PNG:

static BufferedImage createImage4(int w, int h) {
        int[] cmap = new int[16];
        for (int i = 0; i < 16; i++)
                cmap[i] = (i + (i << 4)) << 24;
        IndexColorModel cm = new IndexColorModel(4, 16, cmap, 0, true, 0, DataBuffer.TYPE_BYTE);

        return new BufferedImage(w, h, BufferedImage.TYPE_BYTE_BINARY, cm);

One might notice that the colour palette is actually all black and only the alpha changes - if a browser doesn't support the alpha colourmap then the image will be black. Bummer.

Wikipedia uses 4-bit png's for it's maths equations but I think it only has a transparent colour - and in any event they clearly only work if the background colour of the browser is white. Starting at fully white pages for 10+ hours pre day just burns your eyes out so I force my browser to roughtly amiga-console-grey because that's a colour someone actually thought about before using it. I think we can 'thank' microsoft for the brilliant white background in browsers as before IE they weren't so stupid to choose it as the default. White on black isn't much better either.

But as a result this is the sort of fucked up crap I get out of wikipedia unless I disable my style overrides:

I've started exporting it's pages to PDF so I can actually read them (using a customised mupdf which uses a grey background) but it's formatting leaves a little to be desired and if anything by making it appear like a paper it just emphasises any (of the many) shortcomings in the information content and presentation.

Pretty much any site with maths is pretty shit for that matter; everything from missing or low-quality 2-bit renders to fat javascript libraries that do the layout client-side. Dunno if this approach will be much better but I'm not going to need it very often anyway.

For various reasons from the weather to health to work I've been feeling pretty flat lately and it had me thinking about the past a bit. To think that i've gone from hand-coding raster interrupts and sprite multiplexors to writing information serving software "for fun" is pretty depressing. Computers used to be a much much more fun hobby.

Tagged hacking, java, wanki.
Tuesday, 20 January 2015, 13:01

Yay, NBN is here at last.

Got the nbn hooked up on Friday - and fortunately i'm still in an area they're doing fibre to the home even if it is coming in over-head. Of course most of the country should be getting that too but the total fuckwits running the country decided to show their complete lack of intelligence and add a ton of future cost and grief by changing it to 'fibre to the node' so they can maintain 50 year old copper pairs until they need to go back to the original plan at double the expense, ... but i digress.

It was something i was kind of excited for a few years ago but then I kind of cooled on the idea since the internet is mostly just a pointless waste of time. But then again so is most human activity isn't it? The NBN is mostly just going to be used for ip-tv i'm sure.

The ADSL I had before wasn't too bad so for the most part there isn't much difference so far since most sites were bandwidth limited their end rather than mine but it should be more reliable during wet weather if nothing else (the old modem was getting flakey too and needed a weekly-or-so reboot). I guess the peak (i'm willing to pay for right now) is 2x download and much higher upload (10x or 20x, not sure what it was before) compared to my ADSL, which is better than a poke in the eye.

I am getting a fixed ip address this time and will be setting up a local webserver - mostly because I can and want to but also to play with some web software and move away from google's advertising-supported services. I'm going to try to write my own cms/server/thing based on some experiments I did a few years ago; but it could be a while before I get anywhere because i'm just not in any rush and the weather is too nice over summer (and it's a fairly large undertaking together with me being a bit rusty on the technologies involved).

I'm also going to try to run it on my beagleboard xm too just for kicks - after a bit of a search i found where i'd left it and it seems to be working fine. It should be fast enough for the "expected load".

I'm too lazy to take a picture of it right now but yesterday I finally worked out how to fit the beagleboard and a usb harddrive into a tidy & compact case. I have an old/dead 3.5" USB HDD enclosure made of extruded aluminium that I cut out and filed down some holes for the usb/network slots and had a portable 2.5" HDD I filed down a bit to slide into the other end (retaining all the shock mounting and external case). I have the connecting cable running externally but since it's already got a network cable coming out the same side it sort of "works" and is a lot neater than anything I could come up with when trying to shoehorn the cables into any other reasonably sized box (usb cables are so bulky in a confined area).

I didn't add any holes for the audio/video panel but there should be room for some right-angle plugs should I need it. I guess if i ever replace my burnt out amp I could set it up near that and use it for a radio too.

Tagged beagle, biographical, wanki.
Copyright (C) 2018 Michael Zucchi, All Rights Reserved.Powered by gcc & me!