About

Michael Zucchi

 B.E. (Comp. Sys. Eng.)

Tags

android (44)
beagle (63)
biographical (83)
business (1)
code (57)
cooking (30)
dez (6)
dusk (30)
ffts (3)
forth (3)
free software (4)
games (32)
gloat (2)
globalisation (1)
gnu (4)
graphics (16)
gsoc (4)
hacking (417)
haiku (2)
horticulture (10)
house (23)
hsa (6)
humour (7)
imagez (28)
java (216)
java ee (3)
javafx (48)
jjmpeg (67)
junk (3)
kobo (15)
linux (5)
mediaz (27)
ml (15)
nativez (3)
opencl (117)
os (17)
parallella (97)
pdfz (8)
philosophy (26)
picfx (2)
playerz (1)
politics (7)
ps3 (12)
puppybits (17)
rants (134)
readerz (8)
rez (1)
socles (36)
termz (3)
videoz (6)
wanki (3)
workshop (3)
zedzone (14)
Sunday, 30 December 2018, 08:56

Monitoring Removable USB Drives

I've been looking into writing some media player software and one function I want is to be able to detect inserted drives and do things with them.

inotify

First I looked at using inotify and just monitoring /dev. This is quite simple; you get IN_CREATE and IN_DELETE events when the directories are created or otherwise, and then you can do what is necessary.

Then came the problem of the doing bit - well you want to mount drives. You can use usbmount or udevd but at this point I was trying for an in-memory solution and I couldn't work out how to find out when they were mounted without having another script run.

libblkid & mount(2)

blkid is used by the usbmount script to determine if the device is a mountable partition; so I looked into libblkdid to do the same from code. It's a bit of a quirky api, and is poorly documented but not terribly difficult to use for what I want. The api is designed for implementing glue between kernel and scripts. The variables you get are again, pretty much undocumented as far as i can gather so it's just a process of run it and see what looks useful.

But using it isn't hard. Basically when a /dev/sd* appears I perform a probe and if it has the right variables set I call mount. I only want read-only access so that simplifies the mount options considerably.

Here's a partial prototype example of the inotify handler and mounting process. This is basically what usbmount does but it ignores any options in fstab and so forth.

static int mount_add(const char *name, blkid_probe p) {
        int idx = mount_alloc();

        if (idx >= 0) {
                int res;
                const char *type;
                size_t type_size;

                printf("checking mount %s\n", name);          
                if (blkid_probe_lookup_value(p, "TYPE", &type, &type_size) == 0) {
                        char point[(strlen("/var/run/usb-00") + 1)];

                        sprintf(point, "/var/run/usb-%02d", idx);
                        res = mkdir(point, 0777);
                        res = mount(name, point, type, MS_NOEXEC | MS_RDONLY | MS_NOSUID, NULL);
                        if (res != 0) {
                                perror("mount");
                                return -1;
                        }
                        printf("mounted: %s <- %s\n", point, name);
                        mounted[idx].name = strdup(name);
                        mounted[idx].point = strdup(point);

                        if (blkid_probe_lookup_value(p, "UUID", &type, &type_size) == 0)
                                mounted[idx].uuid = strdup(type);
                } else {
                        idx = -1;
                }
        }
        return idx;
}

static int mount_remove(const char *name) {
        int idx = mount_index(name);
        
        if (idx >= 0) {
                int res = umount2(mounted[idx].point, MNT_DETACH);
                if (res != 0)
                        perror("umount2");
                else
                        printf("unmounted: %s <- %s\n", mounted[idx].point, mounted[idx].name);

                free(mounted[idx].name);
                free(mounted[idx].point);
                free(mounted[idx].uuid);
                memset(&mounted[idx], 0, sizeof(struct mounted));
        }

        return idx;
}

static void
handle_events(int fd, int wd)
{
        char buf[4096]  __attribute__ ((aligned(__alignof__(struct inotify_event))));
        int i;
        ssize_t len;
        char *ptr;

        for (;;) {
                len = read(fd, buf, sizeof buf);
                if (len == -1 && errno != EAGAIN) {
                        perror("read");
                        exit(EXIT_FAILURE);
                }
                if (len <= 0)
                        break;

                for (ptr = buf; ptr < buf + len;) {
                        const struct inotify_event *event;
                        
                        event = (const struct inotify_event *) ptr;

                        if (event->len && strncmp(event->name, "sd", 2) == 0) {
                                char name[(event->len + 1 + strlen("/dev/"))];

                                sprintf(name, "/dev/%s", event->name);
                                
                                if (event->mask & IN_CREATE) {
                                        blkid_probe p = blkid_new_probe_from_filename(name);
                                        int rc;

                                        blkid_probe_enable_partitions(p, 0);
                                        blkid_probe_enable_superblocks(p, 1);
                                        rc = blkid_do_safeprobe(p);
                                        
                                        int idx = mount_add(name, p);
                                        if (idx >= 0) {
                                                // do something with added mount
                                        }
                                        blkid_free_probe(p);
                                }
                                if (event->mask & IN_DELETE) {
                                        int idx = mount_index(name);
                                        if (idx >= 0) {
                                                // do something with removed mount
                                        }
                                }
                        }
                        
                        ptr += sizeof(struct inotify_event) + event->len;
                }
        }
}

udevd, NETLINK_KOBJECT_UEVENT

But I was curious - just how are the dev entries created? I thought it must be udevd or something. The only articles or documentation I could find were unclear on the matter.

So first, NETLINK_KOBJECT_UEVENT, this unpleasantly named 'network' protocol is used these days to monitor kernel activities like usb hotplugging. So I took a small example and started dumping out what I got.

The documentation and examples talk about using a 'struct nmlsghdr' as the datagram payload, but this is in error. You just get the payload directly. And the payload appears to be list of NUL terminated strings; which surprise are just environment variable declarations (i'm noticing a trend here).

Anyway, here's a complete example which dumps out the messages.

#include <sys/types.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <linux/netlink.h>
#include <unistd.h>

#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <ctype.h>

int main(int argc, char **argv) {
        int s = socket(AF_NETLINK, SOCK_DGRAM, NETLINK_KOBJECT_UEVENT);
        struct sockaddr_nl addr = {
                .nl_family = AF_NETLINK,
                .nl_pid = getpid(),
                .nl_groups = ~0
        };
        int res;
        char data[8192];
        struct msghdr msg;
        struct iovec iov;

        bind(s, (void *)&addr, sizeof(addr));
        
        iov.iov_base = data;
        iov.iov_len = 8192;

        msg.msg_name = (void *)&(addr);
        msg.msg_namelen = sizeof(addr);
        msg.msg_iov = &iov;
        msg.msg_iovlen = 1;

        while ((res = recvmsg(s, &msg, 0)) >= 0) {
                if (strcmp(data, "libudev") != 0) {
                        unsigned char *x = data, *e = data+res;

                        while (x < e) {
                                printf(" %s\n", x);
                                x += strlen(x) + 1;
                        }
                        printf("\n");
                } else {
                        for (int i=0;i<res;i+=16) {
                                for (int j=i;j < i+16;j++)
                                        printf(j < res ? " %02x" : "   ", data[j]);
                                printf("  ");
                                for (int j=i;j < i+16;j++)
                                        putchar(j < res ? (isprint(data[j]) ? data[j] : '.') : ' ');
                                printf("\n");
                        }
                        printf("\n");
                }
        }
        
        return 0;
}

The libudev test is because udevd repackages the requests and sends them out again in some proprietary binary format for some reason which is unclear to me at this time.

And this is the output when I plug in a usb drive.

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2
 SUBSYSTEM=usb
 MAJOR=189
 MINOR=645
 DEVNAME=bus/usb/006/006
 DEVTYPE=usb_device
 PRODUCT=90c/1000/1100
 TYPE=0/0/0
 BUSNUM=006
 DEVNUM=006
 SEQNUM=7439

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0
 SUBSYSTEM=usb
 DEVTYPE=usb_interface
 PRODUCT=90c/1000/1100
 TYPE=0/0/0
 INTERFACE=8/6/80
 MODALIAS=usb:v090Cp1000d1100dc00dsc00dp00ic08isc06ip50in00
 SEQNUM=7440

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6
 SUBSYSTEM=scsi
 DEVTYPE=scsi_host
 SEQNUM=7441

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/scsi_host/host6
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/scsi_host/host6
 SUBSYSTEM=scsi_host
 SEQNUM=7442

 bind@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0
 ACTION=bind
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0
 SUBSYSTEM=usb
 DEVTYPE=usb_interface
 DRIVER=usb-storage
 PRODUCT=90c/1000/1100
 TYPE=0/0/0
 INTERFACE=8/6/80
 MODALIAS=usb:v090Cp1000d1100dc00dsc00dp00ic08isc06ip50in00
 SEQNUM=7443

 bind@/devices/pci0000:00/0000:00:10.0/usb6/6-2
 ACTION=bind
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2
 SUBSYSTEM=usb
 MAJOR=189
 MINOR=645
 DEVNAME=bus/usb/006/006
 DEVTYPE=usb_device
 DRIVER=usb
 PRODUCT=90c/1000/1100
 TYPE=0/0/0
 BUSNUM=006
 DEVNUM=006
 SEQNUM=7444

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 89 02 00 00 05 77 c5 e5 27 f8 f5 0c  (........w..'...
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 00 53 55 42 53 59 53 54 45 4d 3d 75  /6-2.SUBSYSTEM=u
 73 62 00 44 45 56 4e 41 4d 45 3d 2f 64 65 76 2f  sb.DEVNAME=/dev/
 62 75 73 2f 75 73 62 2f 30 30 36 2f 30 30 36 00  bus/usb/006/006.
 44 45 56 54 59 50 45 3d 75 73 62 5f 64 65 76 69  DEVTYPE=usb_devi
 63 65 00 50 52 4f 44 55 43 54 3d 39 30 63 2f 31  ce.PRODUCT=90c/1
 30 30 30 2f 31 31 30 30 00 54 59 50 45 3d 30 2f  000/1100.TYPE=0/
 30 2f 30 00 42 55 53 4e 55 4d 3d 30 30 36 00 44  0/0.BUSNUM=006.D
 45 56 4e 55 4d 3d 30 30 36 00 53 45 51 4e 55 4d  EVNUM=006.SEQNUM
 3d 37 34 33 39 00 4d 41 4a 4f 52 3d 31 38 39 00  =7439.MAJOR=189.
 4d 49 4e 4f 52 3d 36 34 35 00 55 53 45 43 5f 49  MINOR=645.USEC_I
 4e 49 54 49 41 4c 49 5a 45 44 3d 32 30 35 35 39  NITIALIZED=20559
 39 39 33 30 37 35 34 35 00 49 44 5f 56 45 4e 44  99307545.ID_VEND
 4f 52 3d 53 61 6d 73 75 6e 67 00 49 44 5f 56 45  OR=Samsung.ID_VE
 4e 44 4f 52 5f 45 4e 43 3d 53 61 6d 73 75 6e 67  NDOR_ENC=Samsung
 00 49 44 5f 56 45 4e 44 4f 52 5f 49 44 3d 30 39  .ID_VENDOR_ID=09
 30 63 00 49 44 5f 4d 4f 44 45 4c 3d 46 6c 61 73  0c.ID_MODEL=Flas
 68 5f 44 72 69 76 65 5f 44 55 4f 00 49 44 5f 4d  h_Drive_DUO.ID_M
 4f 44 45 4c 5f 45 4e 43 3d 46 6c 61 73 68 5c 78  ODEL_ENC=Flash\x
 32 30 44 72 69 76 65 5c 78 32 30 44 55 4f 00 49  20Drive\x20DUO.I
 44 5f 4d 4f 44 45 4c 5f 49 44 3d 31 30 30 30 00  D_MODEL_ID=1000.
 49 44 5f 52 45 56 49 53 49 4f 4e 3d 31 31 30 30  ID_REVISION=1100
 00 49 44 5f 53 45 52 49 41 4c 3d 53 61 6d 73 75  .ID_SERIAL=Samsu
 6e 67 5f 46 6c 61 73 68 5f 44 72 69 76 65 5f 44  ng_Flash_Drive_D
 55 4f 5f 30 33 33 31 35 31 36 30 37 30 30 31 38  UO_0331516070018
 38 31 31 00 49 44 5f 53 45 52 49 41 4c 5f 53 48  811.ID_SERIAL_SH
 4f 52 54 3d 30 33 33 31 35 31 36 30 37 30 30 31  ORT=033151607001
 38 38 31 31 00 49 44 5f 42 55 53 3d 75 73 62 00  8811.ID_BUS=usb.
 49 44 5f 55 53 42 5f 49 4e 54 45 52 46 41 43 45  ID_USB_INTERFACE
 53 3d 3a 30 38 30 36 35 30 3a 00 49 44 5f 56 45  S=:080650:.ID_VE
 4e 44 4f 52 5f 46 52 4f 4d 5f 44 41 54 41 42 41  NDOR_FROM_DATABA
 53 45 3d 53 69 6c 69 63 6f 6e 20 4d 6f 74 69 6f  SE=Silicon Motio
 6e 2c 20 49 6e 63 2e 20 2d 20 54 61 69 77 61 6e  n, Inc. - Taiwan
 20 28 66 6f 72 6d 65 72 6c 79 20 46 65 69 79 61   (formerly Feiya
 20 54 65 63 68 6e 6f 6c 6f 67 79 20 43 6f 72 70   Technology Corp
 2e 29 00 49 44 5f 4d 4f 44 45 4c 5f 46 52 4f 4d  .).ID_MODEL_FROM
 5f 44 41 54 41 42 41 53 45 3d 46 6c 61 73 68 20  _DATABASE=Flash 
 44 72 69 76 65 00 44 52 49 56 45 52 3d 75 73 62  Drive.DRIVER=usb
 00                                               .               

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 8f 01 00 00 05 77 c5 e5 b1 02 47 65  (........w....Ge
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 00 53 55 42  /6-2/6-2:1.0.SUB
 53 59 53 54 45 4d 3d 75 73 62 00 44 45 56 54 59  SYSTEM=usb.DEVTY
 50 45 3d 75 73 62 5f 69 6e 74 65 72 66 61 63 65  PE=usb_interface
 00 50 52 4f 44 55 43 54 3d 39 30 63 2f 31 30 30  .PRODUCT=90c/100
 30 2f 31 31 30 30 00 54 59 50 45 3d 30 2f 30 2f  0/1100.TYPE=0/0/
 30 00 49 4e 54 45 52 46 41 43 45 3d 38 2f 36 2f  0.INTERFACE=8/6/
 38 30 00 4d 4f 44 41 4c 49 41 53 3d 75 73 62 3a  80.MODALIAS=usb:
 76 30 39 30 43 70 31 30 30 30 64 31 31 30 30 64  v090Cp1000d1100d
 63 30 30 64 73 63 30 30 64 70 30 30 69 63 30 38  c00dsc00dp00ic08
 69 73 63 30 36 69 70 35 30 69 6e 30 30 00 53 45  isc06ip50in00.SE
 51 4e 55 4d 3d 37 34 34 30 00 55 53 45 43 5f 49  QNUM=7440.USEC_I
 4e 49 54 49 41 4c 49 5a 45 44 3d 32 30 35 35 39  NITIALIZED=20559
 39 39 33 30 39 32 38 36 00 49 44 5f 56 45 4e 44  99309286.ID_VEND
 4f 52 5f 46 52 4f 4d 5f 44 41 54 41 42 41 53 45  OR_FROM_DATABASE
 3d 53 69 6c 69 63 6f 6e 20 4d 6f 74 69 6f 6e 2c  =Silicon Motion,
 20 49 6e 63 2e 20 2d 20 54 61 69 77 61 6e 20 28   Inc. - Taiwan (
 66 6f 72 6d 65 72 6c 79 20 46 65 69 79 61 20 54  formerly Feiya T
 65 63 68 6e 6f 6c 6f 67 79 20 43 6f 72 70 2e 29  echnology Corp.)
 00 49 44 5f 4d 4f 44 45 4c 5f 46 52 4f 4d 5f 44  .ID_MODEL_FROM_D
 41 54 41 42 41 53 45 3d 46 6c 61 73 68 20 44 72  ATABASE=Flash Dr
 69 76 65 00 44 52 49 56 45 52 3d 75 73 62 2d 73  ive.DRIVER=usb-s
 74 6f 72 61 67 65 00                             torage.         

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 97 00 00 00 29 20 1a 28 a0 05 fb 58  (.......) .(...X
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 00 53 55 42 53 59 53 54 45 4d 3d 73 63 73  t6.SUBSYSTEM=scs
 69 00 44 45 56 54 59 50 45 3d 73 63 73 69 5f 68  i.DEVTYPE=scsi_h
 6f 73 74 00 53 45 51 4e 55 4d 3d 37 34 34 31 00  ost.SEQNUM=7441.
 55 53 45 43 5f 49 4e 49 54 49 41 4c 49 5a 45 44  USEC_INITIALIZED
 3d 32 30 35 35 39 39 39 33 31 30 38 39 37 00     =2055999310897. 

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 9a 00 00 00 a0 05 fb 58 00 00 00 00  (..........X....
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 2f 73 63 73 69 5f 68 6f 73 74 2f 68 6f 73  t6/scsi_host/hos
 74 36 00 53 55 42 53 59 53 54 45 4d 3d 73 63 73  t6.SUBSYSTEM=scs
 69 5f 68 6f 73 74 00 53 45 51 4e 55 4d 3d 37 34  i_host.SEQNUM=74
 34 32 00 55 53 45 43 5f 49 4e 49 54 49 41 4c 49  42.USEC_INITIALI
 5a 45 44 3d 32 30 35 35 39 39 39 33 31 31 38 36  ZED=205599931186
 31 00                                            1.              

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0
 SUBSYSTEM=scsi
 DEVTYPE=scsi_target
 SEQNUM=7445

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0
 SUBSYSTEM=scsi
 DEVTYPE=scsi_device
 MODALIAS=scsi:t-0x00
 SEQNUM=7446

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/scsi_disk/6:0:0:0
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/scsi_disk/6:0:0:0
 SUBSYSTEM=scsi_disk
 SEQNUM=7447

 bind@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0
 ACTION=bind
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0
 SUBSYSTEM=scsi
 DEVTYPE=scsi_device
 DRIVER=sd
 MODALIAS=scsi:t-0x00
 SEQNUM=7448

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/scsi_device/6:0:0:0
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/scsi_device/6:0:0:0
 SUBSYSTEM=scsi_device
 SEQNUM=7449

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/scsi_generic/sg1
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/scsi_generic/sg1
 SUBSYSTEM=scsi_generic
 MAJOR=21
 MINOR=1
 DEVNAME=sg1
 SEQNUM=7450

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/bsg/6:0:0:0
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/bsg/6:0:0:0
 SUBSYSTEM=bsg
 MAJOR=246
 MINOR=1
 DEVNAME=bsg/6:0:0:0
 SEQNUM=7451

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 a5 00 00 00 29 20 1a 28 f2 6a 9f 62  (.......) .(.j.b
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 2f 74 61 72 67 65 74 36 3a 30 3a 30 00 53  t6/target6:0:0.S
 55 42 53 59 53 54 45 4d 3d 73 63 73 69 00 44 45  UBSYSTEM=scsi.DE
 56 54 59 50 45 3d 73 63 73 69 5f 74 61 72 67 65  VTYPE=scsi_targe
 74 00 53 45 51 4e 55 4d 3d 37 34 34 35 00 55 53  t.SEQNUM=7445.US
 45 43 5f 49 4e 49 54 49 41 4c 49 5a 45 44 3d 32  EC_INITIALIZED=2
 30 35 36 30 30 30 36 35 39 36 37 32 00           056000659672.   

 add@/devices/virtual/bdi/8:16
 ACTION=add
 DEVPATH=/devices/virtual/bdi/8:16
 SUBSYSTEM=bdi
 SEQNUM=7452

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 cc 00 00 00 29 20 1a 28 7d b5 06 b5  (.......) .(}...
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 2f 74 61 72 67 65 74 36 3a 30 3a 30 2f 36  t6/target6:0:0/6
 3a 30 3a 30 3a 30 00 53 55 42 53 59 53 54 45 4d  :0:0:0.SUBSYSTEM
 3d 73 63 73 69 00 44 45 56 54 59 50 45 3d 73 63  =scsi.DEVTYPE=sc
 73 69 5f 64 65 76 69 63 65 00 4d 4f 44 41 4c 49  si_device.MODALI
 41 53 3d 73 63 73 69 3a 74 2d 30 78 30 30 00 53  AS=scsi:t-0x00.S
 45 51 4e 55 4d 3d 37 34 34 36 00 55 53 45 43 5f  EQNUM=7446.USEC_
 49 4e 49 54 49 41 4c 49 5a 45 44 3d 32 30 35 36  INITIALIZED=2056
 30 30 30 36 36 31 36 31 35 00 44 52 49 56 45 52  000661615.DRIVER
 3d 73 64 00                                      =sd.            

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 66 00 00 00 2a 18 84 a1 00 00 00 00  (...f...*.......
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 76 69 72 74 75 61 6c 2f 62 64 69 2f  ces/virtual/bdi/
 38 3a 31 36 00 53 55 42 53 59 53 54 45 4d 3d 62  8:16.SUBSYSTEM=b
 64 69 00 53 45 51 4e 55 4d 3d 37 34 35 32 00 55  di.SEQNUM=7452.U
 53 45 43 5f 49 4e 49 54 49 41 4c 49 5a 45 44 3d  SEC_INITIALIZED=
 32 30 35 36 30 30 30 36 36 32 34 30 31 00        2056000662401.  

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 b4 00 00 00 7d b5 06 b5 00 00 00 00  (.......}.......
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 2f 74 61 72 67 65 74 36 3a 30 3a 30 2f 36  t6/target6:0:0/6
 3a 30 3a 30 3a 30 2f 73 63 73 69 5f 64 65 76 69  :0:0:0/scsi_devi
 63 65 2f 36 3a 30 3a 30 3a 30 00 53 55 42 53 59  ce/6:0:0:0.SUBSY
 53 54 45 4d 3d 73 63 73 69 5f 64 65 76 69 63 65  STEM=scsi_device
 00 53 45 51 4e 55 4d 3d 37 34 34 39 00 55 53 45  .SEQNUM=7449.USE
 43 5f 49 4e 49 54 49 41 4c 49 5a 45 44 3d 32 30  C_INITIALIZED=20
 35 36 30 30 30 36 36 35 31 34 31 00              56000665141.    

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/block/sdb
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/block/sdb
 SUBSYSTEM=block
 MAJOR=8
 MINOR=16
 DEVNAME=sdb
 DEVTYPE=disk
 SEQNUM=7453

 add@/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/block/sdb/sdb1
 ACTION=add
 DEVPATH=/devices/pci0000:00/0000:00:10.0/usb6/6-2/6-2:1.0/host6/target6:0:0/6:0:0:0/block/sdb/sdb1
 SUBSYSTEM=block
 MAJOR=8
 MINOR=17
 DEVNAME=sdb1
 DEVTYPE=partition
 PARTN=1
 SEQNUM=7454

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 b0 00 00 00 89 0e a9 0f 00 00 00 00  (...............
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 2f 74 61 72 67 65 74 36 3a 30 3a 30 2f 36  t6/target6:0:0/6
 3a 30 3a 30 3a 30 2f 73 63 73 69 5f 64 69 73 6b  :0:0:0/scsi_disk
 2f 36 3a 30 3a 30 3a 30 00 53 55 42 53 59 53 54  /6:0:0:0.SUBSYST
 45 4d 3d 73 63 73 69 5f 64 69 73 6b 00 53 45 51  EM=scsi_disk.SEQ
 4e 55 4d 3d 37 34 34 37 00 55 53 45 43 5f 49 4e  NUM=7447.USEC_IN
 49 54 49 41 4c 49 5a 45 44 3d 32 30 35 36 30 30  ITIALIZED=205600
 30 36 36 36 30 33 30 00                          0666030.        

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 d4 00 00 00 3c e8 b1 f8 00 00 00 00  (.......<.......
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 2f 74 61 72 67 65 74 36 3a 30 3a 30 2f 36  t6/target6:0:0/6
 3a 30 3a 30 3a 30 2f 73 63 73 69 5f 67 65 6e 65  :0:0:0/scsi_gene
 72 69 63 2f 73 67 31 00 53 55 42 53 59 53 54 45  ric/sg1.SUBSYSTE
 4d 3d 73 63 73 69 5f 67 65 6e 65 72 69 63 00 44  M=scsi_generic.D
 45 56 4e 41 4d 45 3d 2f 64 65 76 2f 73 67 31 00  EVNAME=/dev/sg1.
 53 45 51 4e 55 4d 3d 37 34 35 30 00 4d 41 4a 4f  SEQNUM=7450.MAJO
 52 3d 32 31 00 4d 49 4e 4f 52 3d 31 00 55 53 45  R=21.MINOR=1.USE
 43 5f 49 4e 49 54 49 41 4c 49 5a 45 44 3d 32 30  C_INITIALIZED=20
 35 36 30 30 30 36 36 36 35 39 35 00              56000666595.    

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 cf 00 00 00 0d df d8 a1 00 00 00 00  (...............
 00 00 00 00 00 00 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 2f 74 61 72 67 65 74 36 3a 30 3a 30 2f 36  t6/target6:0:0/6
 3a 30 3a 30 3a 30 2f 62 73 67 2f 36 3a 30 3a 30  :0:0:0/bsg/6:0:0
 3a 30 00 53 55 42 53 59 53 54 45 4d 3d 62 73 67  :0.SUBSYSTEM=bsg
 00 44 45 56 4e 41 4d 45 3d 2f 64 65 76 2f 62 73  .DEVNAME=/dev/bs
 67 2f 36 3a 30 3a 30 3a 30 00 53 45 51 4e 55 4d  g/6:0:0:0.SEQNUM
 3d 37 34 35 31 00 4d 41 4a 4f 52 3d 32 34 36 00  =7451.MAJOR=246.
 4d 49 4e 4f 52 3d 31 00 55 53 45 43 5f 49 4e 49  MINOR=1.USEC_INI
 54 49 41 4c 49 5a 45 44 3d 32 30 35 36 30 30 30  TIALIZED=2056000
 36 36 37 30 34 32 00                             667042.         

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 54 03 00 00 f0 03 1d b7 7b cb c5 ee  (...T.......{...
 02 00 04 00 10 80 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 2f 74 61 72 67 65 74 36 3a 30 3a 30 2f 36  t6/target6:0:0/6
 3a 30 3a 30 3a 30 2f 62 6c 6f 63 6b 2f 73 64 62  :0:0:0/block/sdb
 00 53 55 42 53 59 53 54 45 4d 3d 62 6c 6f 63 6b  .SUBSYSTEM=block
 00 44 45 56 4e 41 4d 45 3d 2f 64 65 76 2f 73 64  .DEVNAME=/dev/sd
 62 00 44 45 56 54 59 50 45 3d 64 69 73 6b 00 53  b.DEVTYPE=disk.S
 45 51 4e 55 4d 3d 37 34 35 33 00 55 53 45 43 5f  EQNUM=7453.USEC_
 49 4e 49 54 49 41 4c 49 5a 45 44 3d 32 30 35 36  INITIALIZED=2056
 30 30 30 37 33 36 33 35 39 00 4d 41 4a 4f 52 3d  000736359.MAJOR=
 38 00 4d 49 4e 4f 52 3d 31 36 00 49 44 5f 56 45  8.MINOR=16.ID_VE
 4e 44 4f 52 3d 53 61 6d 73 75 6e 67 00 49 44 5f  NDOR=Samsung.ID_
 56 45 4e 44 4f 52 5f 45 4e 43 3d 53 61 6d 73 75  VENDOR_ENC=Samsu
 6e 67 5c 78 32 30 00 49 44 5f 56 45 4e 44 4f 52  ng\x20.ID_VENDOR
 5f 49 44 3d 30 39 30 63 00 49 44 5f 4d 4f 44 45  _ID=090c.ID_MODE
 4c 3d 46 6c 61 73 68 5f 44 72 69 76 65 5f 44 55  L=Flash_Drive_DU
 4f 00 49 44 5f 4d 4f 44 45 4c 5f 45 4e 43 3d 46  O.ID_MODEL_ENC=F
 6c 61 73 68 5c 78 32 30 44 72 69 76 65 5c 78 32  lash\x20Drive\x2
 30 44 55 4f 5c 78 32 30 00 49 44 5f 4d 4f 44 45  0DUO\x20.ID_MODE
 4c 5f 49 44 3d 31 30 30 30 00 49 44 5f 52 45 56  L_ID=1000.ID_REV
 49 53 49 4f 4e 3d 31 31 30 30 00 49 44 5f 53 45  ISION=1100.ID_SE
 52 49 41 4c 3d 53 61 6d 73 75 6e 67 5f 46 6c 61  RIAL=Samsung_Fla
 73 68 5f 44 72 69 76 65 5f 44 55 4f 5f 30 33 33  sh_Drive_DUO_033
 31 35 31 36 30 37 30 30 31 38 38 31 31 2d 30 3a  1516070018811-0:
 30 00 49 44 5f 53 45 52 49 41 4c 5f 53 48 4f 52  0.ID_SERIAL_SHOR
 54 3d 30 33 33 31 35 31 36 30 37 30 30 31 38 38  T=03315160700188
 31 31 00 49 44 5f 54 59 50 45 3d 64 69 73 6b 00  11.ID_TYPE=disk.
 49 44 5f 49 4e 53 54 41 4e 43 45 3d 30 3a 30 00  ID_INSTANCE=0:0.
 49 44 5f 42 55 53 3d 75 73 62 00 49 44 5f 55 53  ID_BUS=usb.ID_US
 42 5f 49 4e 54 45 52 46 41 43 45 53 3d 3a 30 38  B_INTERFACES=:08
 30 36 35 30 3a 00 49 44 5f 55 53 42 5f 49 4e 54  0650:.ID_USB_INT
 45 52 46 41 43 45 5f 4e 55 4d 3d 30 30 00 49 44  ERFACE_NUM=00.ID
 5f 55 53 42 5f 44 52 49 56 45 52 3d 75 73 62 2d  _USB_DRIVER=usb-
 73 74 6f 72 61 67 65 00 44 45 56 4c 49 4e 4b 53  storage.DEVLINKS
 3d 2f 64 65 76 2f 64 69 73 6b 2f 62 79 2d 69 64  =/dev/disk/by-id
 2f 75 73 62 2d 53 61 6d 73 75 6e 67 5f 46 6c 61  /usb-Samsung_Fla
 73 68 5f 44 72 69 76 65 5f 44 55 4f 5f 30 33 33  sh_Drive_DUO_033
 31 35 31 36 30 37 30 30 31 38 38 31 31 2d 30 3a  1516070018811-0:
 30 20 2f 64 65 76 2f 64 69 73 6b 2f 62 79 2d 70  0 /dev/disk/by-p
 61 74 68 2f 70 63 69 2d 30 30 30 30 3a 30 30 3a  ath/pci-0000:00:
 31 30 2e 30 2d 75 73 62 2d 30 3a 32 3a 31 2e 30  10.0-usb-0:2:1.0
 2d 73 63 73 69 2d 30 3a 30 3a 30 3a 30 00 49 44  -scsi-0:0:0:0.ID
 5f 50 41 54 48 3d 70 63 69 2d 30 30 30 30 3a 30  _PATH=pci-0000:0
 30 3a 31 30 2e 30 2d 75 73 62 2d 30 3a 32 3a 31  0:10.0-usb-0:2:1
 2e 30 2d 73 63 73 69 2d 30 3a 30 3a 30 3a 30 00  .0-scsi-0:0:0:0.
 49 44 5f 50 41 54 48 5f 54 41 47 3d 70 63 69 2d  ID_PATH_TAG=pci-
 30 30 30 30 5f 30 30 5f 31 30 5f 30 2d 75 73 62  0000_00_10_0-usb
 2d 30 5f 32 5f 31 5f 30 2d 73 63 73 69 2d 30 5f  -0_2_1_0-scsi-0_
 30 5f 30 5f 30 00 49 44 5f 50 41 52 54 5f 54 41  0_0_0.ID_PART_TA
 42 4c 45 5f 54 59 50 45 3d 64 6f 73 00 54 41 47  BLE_TYPE=dos.TAG
 53 3d 3a 73 79 73 74 65 6d 64 3a 00              S=:systemd:.    

 6c 69 62 75 64 65 76 00 fe ed ca fe 28 00 00 00  libudev.....(...
 28 00 00 00 e5 04 00 00 f0 03 1d b7 cb 23 44 89  (............#D.
 02 00 04 00 10 80 00 00 41 43 54 49 4f 4e 3d 61  ........ACTION=a
 64 64 00 44 45 56 50 41 54 48 3d 2f 64 65 76 69  dd.DEVPATH=/devi
 63 65 73 2f 70 63 69 30 30 30 30 3a 30 30 2f 30  ces/pci0000:00/0
 30 30 30 3a 30 30 3a 31 30 2e 30 2f 75 73 62 36  000:00:10.0/usb6
 2f 36 2d 32 2f 36 2d 32 3a 31 2e 30 2f 68 6f 73  /6-2/6-2:1.0/hos
 74 36 2f 74 61 72 67 65 74 36 3a 30 3a 30 2f 36  t6/target6:0:0/6
 3a 30 3a 30 3a 30 2f 62 6c 6f 63 6b 2f 73 64 62  :0:0:0/block/sdb
 2f 73 64 62 31 00 53 55 42 53 59 53 54 45 4d 3d  /sdb1.SUBSYSTEM=
 62 6c 6f 63 6b 00 44 45 56 4e 41 4d 45 3d 2f 64  block.DEVNAME=/d
 65 76 2f 73 64 62 31 00 44 45 56 54 59 50 45 3d  ev/sdb1.DEVTYPE=
 70 61 72 74 69 74 69 6f 6e 00 50 41 52 54 4e 3d  partition.PARTN=
 31 00 53 45 51 4e 55 4d 3d 37 34 35 34 00 55 53  1.SEQNUM=7454.US
 45 43 5f 49 4e 49 54 49 41 4c 49 5a 45 44 3d 32  EC_INITIALIZED=2
 30 35 36 30 30 30 37 39 35 39 37 39 00 4d 41 4a  056000795979.MAJ
 4f 52 3d 38 00 4d 49 4e 4f 52 3d 31 37 00 49 44  OR=8.MINOR=17.ID
 5f 42 55 53 3d 75 73 62 00 49 44 5f 49 4e 53 54  _BUS=usb.ID_INST
 41 4e 43 45 3d 30 3a 30 00 49 44 5f 4d 4f 44 45  ANCE=0:0.ID_MODE
 4c 3d 46 6c 61 73 68 5f 44 72 69 76 65 5f 44 55  L=Flash_Drive_DU
 4f 00 49 44 5f 4d 4f 44 45 4c 5f 45 4e 43 3d 46  O.ID_MODEL_ENC=F
 6c 61 73 68 5c 78 32 30 44 72 69 76 65 5c 78 32  lash\x20Drive\x2
 30 44 55 4f 5c 78 32 30 00 49 44 5f 4d 4f 44 45  0DUO\x20.ID_MODE
 4c 5f 49 44 3d 31 30 30 30 00 49 44 5f 50 41 52  L_ID=1000.ID_PAR
 54 5f 54 41 42 4c 45 5f 54 59 50 45 3d 64 6f 73  T_TABLE_TYPE=dos
 00 49 44 5f 50 41 54 48 3d 70 63 69 2d 30 30 30  .ID_PATH=pci-000
 30 3a 30 30 3a 31 30 2e 30 2d 75 73 62 2d 30 3a  0:00:10.0-usb-0:
 32 3a 31 2e 30 2d 73 63 73 69 2d 30 3a 30 3a 30  2:1.0-scsi-0:0:0
 3a 30 00 49 44 5f 50 41 54 48 5f 54 41 47 3d 70  :0.ID_PATH_TAG=p
 63 69 2d 30 30 30 30 5f 30 30 5f 31 30 5f 30 2d  ci-0000_00_10_0-
 75 73 62 2d 30 5f 32 5f 31 5f 30 2d 73 63 73 69  usb-0_2_1_0-scsi
 2d 30 5f 30 5f 30 5f 30 00 49 44 5f 52 45 56 49  -0_0_0_0.ID_REVI
 53 49 4f 4e 3d 31 31 30 30 00 49 44 5f 53 45 52  SION=1100.ID_SER
 49 41 4c 3d 53 61 6d 73 75 6e 67 5f 46 6c 61 73  IAL=Samsung_Flas
 68 5f 44 72 69 76 65 5f 44 55 4f 5f 30 33 33 31  h_Drive_DUO_0331
 35 31 36 30 37 30 30 31 38 38 31 31 2d 30 3a 30  516070018811-0:0
 00 49 44 5f 53 45 52 49 41 4c 5f 53 48 4f 52 54  .ID_SERIAL_SHORT
 3d 30 33 33 31 35 31 36 30 37 30 30 31 38 38 31  =033151607001881
 31 00 49 44 5f 54 59 50 45 3d 64 69 73 6b 00 49  1.ID_TYPE=disk.I
 44 5f 55 53 42 5f 44 52 49 56 45 52 3d 75 73 62  D_USB_DRIVER=usb
 2d 73 74 6f 72 61 67 65 00 49 44 5f 55 53 42 5f  -storage.ID_USB_
 49 4e 54 45 52 46 41 43 45 53 3d 3a 30 38 30 36  INTERFACES=:0806
 35 30 3a 00 49 44 5f 55 53 42 5f 49 4e 54 45 52  50:.ID_USB_INTER
 46 41 43 45 5f 4e 55 4d 3d 30 30 00 49 44 5f 56  FACE_NUM=00.ID_V
 45 4e 44 4f 52 3d 53 61 6d 73 75 6e 67 00 49 44  ENDOR=Samsung.ID
 5f 56 45 4e 44 4f 52 5f 45 4e 43 3d 53 61 6d 73  _VENDOR_ENC=Sams
 75 6e 67 5c 78 32 30 00 49 44 5f 56 45 4e 44 4f  ung\x20.ID_VENDO
 52 5f 49 44 3d 30 39 30 63 00 44 45 56 4c 49 4e  R_ID=090c.DEVLIN
 4b 53 3d 2f 64 65 76 2f 64 69 73 6b 2f 62 79 2d  KS=/dev/disk/by-
 6c 61 62 65 6c 2f 53 61 6d 73 75 6e 67 5c 78 32  label/Samsung\x2
 30 55 53 42 20 2f 64 65 76 2f 64 69 73 6b 2f 62  0USB /dev/disk/b
 79 2d 70 61 74 68 2f 70 63 69 2d 30 30 30 30 3a  y-path/pci-0000:
 30 30 3a 31 30 2e 30 2d 75 73 62 2d 30 3a 32 3a  00:10.0-usb-0:2:
 31 2e 30 2d 73 63 73 69 2d 30 3a 30 3a 30 3a 30  1.0-scsi-0:0:0:0
 2d 70 61 72 74 31 20 2f 64 65 76 2f 64 69 73 6b  -part1 /dev/disk
 2f 62 79 2d 69 64 2f 75 73 62 2d 53 61 6d 73 75  /by-id/usb-Samsu
 6e 67 5f 46 6c 61 73 68 5f 44 72 69 76 65 5f 44  ng_Flash_Drive_D
 55 4f 5f 30 33 33 31 35 31 36 30 37 30 30 31 38  UO_0331516070018
 38 31 31 2d 30 3a 30 2d 70 61 72 74 31 20 2f 64  811-0:0-part1 /d
 65 76 2f 64 69 73 6b 2f 62 79 2d 75 75 69 64 2f  ev/disk/by-uuid/
 45 35 34 31 2d 43 30 44 44 00 49 44 5f 46 53 5f  E541-C0DD.ID_FS_
 4c 41 42 45 4c 3d 53 61 6d 73 75 6e 67 5f 55 53  LABEL=Samsung_US
 42 00 49 44 5f 46 53 5f 4c 41 42 45 4c 5f 45 4e  B.ID_FS_LABEL_EN
 43 3d 53 61 6d 73 75 6e 67 5c 78 32 30 55 53 42  C=Samsung\x20USB
 00 49 44 5f 46 53 5f 55 55 49 44 3d 45 35 34 31  .ID_FS_UUID=E541
 2d 43 30 44 44 00 49 44 5f 46 53 5f 55 55 49 44  -C0DD.ID_FS_UUID
 5f 45 4e 43 3d 45 35 34 31 2d 43 30 44 44 00 49  _ENC=E541-C0DD.I
 44 5f 46 53 5f 56 45 52 53 49 4f 4e 3d 46 41 54  D_FS_VERSION=FAT
 33 32 00 49 44 5f 46 53 5f 54 59 50 45 3d 76 66  32.ID_FS_TYPE=vf
 61 74 00 49 44 5f 46 53 5f 55 53 41 47 45 3d 66  at.ID_FS_USAGE=f
 69 6c 65 73 79 73 74 65 6d 00 49 44 5f 50 41 52  ilesystem.ID_PAR
 54 5f 45 4e 54 52 59 5f 53 43 48 45 4d 45 3d 64  T_ENTRY_SCHEME=d
 6f 73 00 49 44 5f 50 41 52 54 5f 45 4e 54 52 59  os.ID_PART_ENTRY
 5f 54 59 50 45 3d 30 78 63 00 49 44 5f 50 41 52  _TYPE=0xc.ID_PAR
 54 5f 45 4e 54 52 59 5f 4e 55 4d 42 45 52 3d 31  T_ENTRY_NUMBER=1
 00 49 44 5f 50 41 52 54 5f 45 4e 54 52 59 5f 4f  .ID_PART_ENTRY_O
 46 46 53 45 54 3d 36 34 00 49 44 5f 50 41 52 54  FFSET=64.ID_PART
 5f 45 4e 54 52 59 5f 53 49 5a 45 3d 31 32 35 33  _ENTRY_SIZE=1253
 30 34 37 36 38 00 49 44 5f 50 41 52 54 5f 45 4e  04768.ID_PART_EN
 54 52 59 5f 44 49 53 4b 3d 38 3a 31 36 00 54 41  TRY_DISK=8:16.TA
 47 53 3d 3a 73 79 73 74 65 6d 64 3a 00           GS=:systemd:.   

Ignoring the libudev snot there's basically a flurry of events, a short pause, and then the last couple of interesting events where the device nodes become available.

AS FAR AS I CAN GATHER ... there appears to be no documentation regarding the format or order of these kernel messages; what variables will be included and what they mean. Well there's always the kernel source but I could find nothing in [kernel-source]/Documnetation nor anywhere else.

I suppose it is relatively straightforward to just 'suck it and see' but documentation would be nice all the same.

At this point I went down a rabbit hole of trying to work out who ACTUALLY creates the /dev entries; it turns out it's the kernel. So infact udevd isn't really doing anything particularly useful on my system. Probably just as well now that the systemd developers have started 'maintaining' it.

systemd

So anyway i've done a bit of prototyping and I'm sure I can get away without needing udevd at all, systemd is thankfully not anywhere near the mele already. Then I went down another rabbit hole of finding out the current status of systemd.

It's still not good. And i'm pretty sure it never will be. With any luck IBM will pull their finger out and realise what a huge risk and waste of money the project is to their new acquisition and throw it in the bin where it belongs. But of course i'm not holding my breath, it's been designed to be unremovable and was agressively promoted and pushed onto 3rd party projects for a reason. As an aside, hopefully IBM can make jfs a first-class filesystem citizen rather than the red-headed step-child it seems to be these days.

The blog fromthecodefront has a very clear analysis of the situation. Grating narcissitic and abusive personalities aside, this is the first time i've come across any sourcode. It is not pretty. It looks exactly like the code I write when i'm developing prototypes; lots of cut and paste, poor modularisation, meaningless function names, etc. Although I usually tend to use more comments, meaningful labels, and fewer weird-arsed macros to 'save typing' (infact i almost never use macros for that anymore; inline or plain functions can usually be used). And because it's all in C, there's quite an impedence mismatch when a scripting language would be the better choice. This is not the type of code I force onto others, and certainly not any system-critical components.

The developers seem to be out of their depth and absolute arseholes when called out on it. The passive-aggressive and other psychopathic behaviours are very troublesome; I dont deal with abusers like that and neither should you. Multiple commenters say that's just ad-hominem and not important: but it is important, and these guys are not nice people.

Fortunately I usually dont have bother with systemd much anyway - I have few services and they are running already. Shutdown and/or suspsend are sometimes broken but I can always just pull the power plug. Asking for passwords on encrypted disks is also pretty shit (console output overflows the prompts); it's just something else I have to workaround.

Actually I tried using it to stop udevd so I could determine what if anything it was doing on my system: and it refused to let me turn it off or disable it (or mask it, whatever they fuck they want call it). I stopped, disabled, and masked every 'service' or 'socket' with udev in the name, no asusage. I had to chmod the systemd-udevd binary to non-executable in the end.

The only saving grace is that systemd-udevd doesn't really do anything so it can't really fuck it up. Even systemd doesn't really do much apart from run a few scripts in some order.

Where's this going

Well i'm not really sure if i'll stick to a simple monolothic music service or go the whole hog and work on a modularised multi-process (and significnalty more secure) design. Simple is nice but the other approach has some relatively interesting things to play with.

But it's all very time-consuming and too sedentry. My body is falling apart.

I need to get out of the house more; every time i get stuck into a coding project (most recently for work) I seem to rapidly stack on weight. It's related to some (medicinal) drugs once taken and now stopped and obviously some over-consumption and it's fucking pissing me off. The weight is also completely fucking up my sleep and that's pissing me off even more. I'm pissed off a lot lately!

Tagged code, hacking.
Friday, 28 December 2018, 08:46

Speaker Boxes and Music Players

I finally got around to mostly finishing the speakers I was making out of recovered drivers. Here's a bit of a photo-diary of the final stages.

Here we have the completed woodwork. It has been glued and sanded square. I'm mostly happy with how it turned out but I could have done a bit more sanding; I was over it by then. In each case I pretty much practiced on one and got better on the other one, hence a few differences here and there. The one on the left is unpainted and the one on the right has a coat of sanding sealer and also the binding post mounting mechanism.

Three undercoats, lots of sanding and two overcoats and it's ready to be put together.

Given the paint was so expensive ($90) I did a bit of a shit job in the end; I just got sick of dealing with everything. First I had a cheap throwaway brush which I should've just thrown away before I even started. Then the paint was too thick for the weather and needed more thinning, then I tried using a small roller - but again i got some cheap junk which left behind piles of dust and shit, and because I hadn't used a roller before I didn't apply it as well as I might have.

The next shot just shows the mounting point for the binding posts.

Even this I kinda fucked up. I tried using the drill press to drill the screw holes to mount it but to avoid scratching the paint I had it on a small block of foam. It moved when I wasn't looking and I drilled it in the wrong spot. It's hidden but I know it's there.

Soldering the wires.

Completed binding posts.

Connecting the driver.

Inserting the acoustic foam. I have 25mm at the read and 12.5mm down the sides - about as much as could fit in the small box.

A completed speaker (so far). The screws (all 24 of them) for the box are really only aesthetic as the box is very well glued. They did help when setting the glue though.

The pair. I've only redone the rubber surround on the right one so far, the one on the left is still a bit scratchy with high volume as a result. The mounting rubber washers/etc is still work in progress.

I'm still yet to get some fabric for the grilles. I did a quick peruse through lincraft but couldn't find any suitable, next time i'll ask and there's a couple of other places to try.

And finally a shot in testing. The plan is to put the electronics in another box with a battery pack. I've got the bits for the battery but am still working on the mechanics.

How do they sound? Well ok. They're not going to break any technical records; there's no engineering behind the boxes other than being a sealed box with a bit of foam in it. Each only has a woofer so the top-end is very retarded. The bottom end isn't too bad but it does have an echoey effect with some mid-tones, then again so do any other small speakers i've tried (including the $1000/pair units I just installed in the dining room). Quite ok for a bit of outside doof though!

It's been a good little project to get away from the screen so far, although finishing it off will not be.

Electronics and Software

I was looking around for hardware for the player but since I have the mele doing nothing I thought I may as well use that. Most of the other prototyping boards around now are either too high powered or missing some necessary bits; although the nanopi boards look quite attractive (on a side-note I can't believe how expensive raspberry pi shit is in australia). As this is nominally a 'portable' system based on recycled speakers (with no tweeters!) the audio quality isn't terribly important, the on-board dac/codec sounds good enough to me. I'm also considering an old phone or an old 7" tablet I have, although it's far less attractive to me if they can only run android (fuck that shit; yeah fuck you too google).

After spending most of a day on it I managed to get debian running on the mele. I tried the sunxi wiki and building my own kernel and bootloader but without any debug console I couldn't tell if it was working or not. I got this server image to work in the end, using the jessie image.

While trying to build using the sunxi instructions I discovered that the ubuntu maintainers have a really broken idea about what a cross-compiler should do. And they're really quite rude about it, hiding their rudeness behind the `code of conduct'.

To avoid dragging in gobs of junk and other poettering snot I built a basic SDL (1.2) and a cut-down ffmpeg (1.0) from source and have a basic console music player working. With a little more work I can control it with the mele airmouse although without a screen it's going to be difficult to do too many interesting things. Something to poke about with.

Tagged hacking, linux, workshop.
Tuesday, 15 May 2018, 19:00

Backend stuff

Winter has hit here and along with insomnia i'm not really feeling like doing much of an evening but i've dabbled a few times and basically ported the Java version of a tree-revision database to C.

At this point i've just got the core done - schema/bindings and most of the client api. I'm pretty sure it's solid but I need to write a lot of testing and validation code to make sure it will be reliable and performant enough, and then write a bunch more to turn it into something interesting.

But i've been at a desk for 10 hours straight and my feet are icy cold so it's not happening tonight.

Tagged hacking, zedzone.
Sunday, 29 April 2018, 12:02

BDB | !BDB?

I mentioned a few posts ago that there doesn't seem to be many NoSQL databases around anymore - at least last time I looked a year or two ago, all the buzz from a decade ago had gone away. Various libraries became proprietary-commercial or got abandoned.

For some reason I can't remember I went looking for BerkeleyDB alternatives and hit this stackoverflow question which points to some of them.

So I guess I was a little mistaken, there are still a few around, but not all are appropriate for what I want it for:

I guess the best of those is LMDB - i'd come across it whilst using Caffe but never looked into it. Given it's roots in replacing BDB it has enough similarities in API and features to be a good match for what I want (and written in a sane language) although a couple of niggles exist such as the lack of sequences and all the fixed-sized structures (and database size). Being a part of a specific project (OpenLDAP) means it's hit maturity without features that might be useful elsewhere.

The multi-version concurrency control and so on is pretty neat anyway. No transaction logs is a good thing. If I ever get time I might play with those ideas a little in Java - not because I necessarily think it's a great idea but just to see if it's possible. I played with an extensible hash thing for indexing in camel many years ago but it was plagued by durability problems.

Back to LMDB - i'll definitely give it a go for my revisioned database thing - at some point.

Tagged hacking, zedzone.
Thursday, 26 April 2018, 09:05

Rabbit Holes All The Way Down

I kept poking around the blog code over the last couple of days. It just keeps leading to more and more questions.

DBD

Tuesday I mostly spent re-brushing up on the C api for Berkeley DB and designing the schema to implement my version database using it. At some point since I last looked foreign key constraints must have been added so I implemented that - unfortunately unlike JE they don't support self-referential keys (where a field references the primary key of the same object) so I will have to code up a couple of cases for that manually. Actually i'm not sure I even need fully indexed key constraints as the database is designed never to have deletions. If I ever get that far i'll do some benchmarking to evaluate the tradeoffs, or decide how to do deletions.

During the journey I also discovered that at some point Berkeley DB JE changed licenses again - it had been AGPL3 last time I looked. Now it's changed to Apache. I wonder if this is another project soon to be abandoned to the ASF? Anyway it doesn't make much difference to my Free Software projects (not that I ever got far enough to publish any) but it'll be handy for work as i've wanted to use it pleny of times. It's about the only decent NoSQL DB left these days.

Uploading JavaScript

I pretty much detest JavaScript but I wanted to look at how to write some sort of web-based editor for writing posts and I don't really feel like writing yet another MIME parser to handle multipart/form. Well I probably will have to eventually (or likely dig one of the few i've already written back up) but in the mean-time I investigated direct uploads using XJAX.

Most results from searching turn up JQuery snot but I eventually found some raw JavaScript using XMLHttpRequest directly. Given it's only a few lines of code one has to wonder about these 'frameworks'. I digress. I played around a bit, extended my FastCGI library to support streaming stdin and wrote a basic REST-like `uploader' that can handle binary blobs directly without any messy protocol parsing. Yay. And then I fell down another hole ... how the fuck am I going to do security?

I don't really want to buy an SSL cert for this site but using a self-signed certificate isn't really any good. Without that pretty much any auth system is wildly insecure. I started looking into JavaScript libraries for crypto - some are a little over the top but there are a few smaller ones that might serve the purpose. Crypto has a lot of gotchas and one can't be an expert in everything so i'm not sure I want to start down what would be a very long and winding road just to post to a website.

So i'm toying with a few ideas. First just do nothing, stick to ssh and emacs for posting. If I ever bother with comments or feedback they can be anonymous and not require auth. Or instead of using JavaScript write a standalone Java editor / operator console that calls REST services. Or even using an ssh driven backend. This has some appeal personally but I'll see. Another is to use SSL + Digest Auth - this way I let the browsers and server handle all the complexity and get a mostly ok system. If I install my own CA on my local browser(s) and enforce client certificates from the server side, it should be reasonably secure.

Damn windy road already.

I need a real rest

My sleep has been particularly bad of late. The sleep apnoae is quite bad and I regularly (mostly) forget to wear the mouth splint which doesn't-treat-it-particularly-well-but-it's-better-than-nothing. At least I remembered last night.

Today I gotta try and do some hours for work though. At the moment i'm trying to decipher some statistical software written in matlab, which is about my most favouritist thing in the whole world. Fuck matlab.

Oh, I also bought some mice. I've got a couple of small 'travel mouse' mice that I much prefer to the standard fare and although they used to be easy to find they've become quite scarce around here. What ever happened to BenQ anyway? All the local retail only have microsoft or logitech or their own badged chinese crap now. Coordless also seem to have taken over (higher margins one suspects). I looked everwhere locally and on the usual suspects online but couldn't find anything decent. Oddly enough the ThinkPad one I already have was one of cheapest, and from the source, so I ordered a couple to tide me over for the forseeable future. On a whim I also added a wireless 'laser' one as well, although it's marginally larger.

Tagged hacking, zedzone.
Tuesday, 24 April 2018, 17:38

FastCGI experiments

It's not particularly important - i'm lucky to get more than one-non-bot hit in a given day - but I thought i'd have a look into FastCGI. If in the future I do use a database backend or even a Java one it should be an easy way to get some performance while leveraging the simplicity of CGI and leaving the protocol stuff to apache.

After a bit of background reading and looking into some 'simple' implementations I decided to just roll my own. The 'official' fastcgi.com site is no longer live so I didn't think it worth playing with the official sdk. The way it handled stdio just seemed a little odd as well.

With the use of a few GNU libc extensions for stdio (cookie streams) and memory (obstacks) I put together enough of a partial (but robust) implementation to serve output-only pages from the fcgid module in a few hundred lines of code.

This is the public api for it.

struct fcgi_param {
        char *name;
        char *value;
};

struct fcgi {
        // Active during cgi request
        FILE *stdout;
        FILE *stderr;

        // Current request info
        unsigned char rid1, rid0;
        unsigned char flags;
        unsigned char role;

        // Current request params (environment)
        size_t param_length;
        size_t param_size;
        struct fcgi_param *param;
        struct obstack param_stack;

        // Internal buffer stuff
        int fd;
        size_t pos;
        size_t limit;
        size_t buffer_size;
        unsigned char *buffer;
};

typedef int (*fcgi_callback_t)(struct fcgi *, void *);

struct fcgi *fcgi_alloc(void);
void fcgi_free(struct fcgi *cgi);

int fcgi_accept_all(struct fcgi *cgi, fcgi_callback_t cb, void *data);
char *fcgi_getenv(struct fcgi *cgi, const char *name);

I didn't bother to implement concurrent requests, the various access control roles, or STDIN messages. The first doesn't appear to be used by mod_fcgi (it handles concurrency itself) and I don't need the rest (yet at least). As previously stated I used GNU libc extensions to implement custom stdio streams for stdout and stderr, although I used a custom 'zero-copy' buffer implementation for the protocol handling (wherein the calls can access the internal buffer address rather than having to copy data around).

Converting a CGI program is a little more involved than using the original SDK because it doesn't hide the i/o behind macros or use global variables to pass information. Instead via a context-specific handle it provides stdio compatible FILE handles and a separate environmental variable lookup function. Of course it is possible to write a handler callback which can implement such a solution.

The main function of a the fast cgi program just allocates the context, calls accept_all and then free. The callback is invoked for each request and can access stdout/stderr from the context using stdio calls as it wishes.

Apache config

Here's the basic apache config snipped I used to hook it into `/blog' on a server (I did this locally rather than live on this site though).

        ScriptAlias /blog /path/fcgi-test.fcgi

        FcgidCmdOptions /path/fcgi-test MaxProcesses 1

        <Directory "/path">
                AllowOverride None
                Options +ExecCGI
                Require all granted
        </Directory>

Custom streams and cookies

Using a GNU extension it is trivial to hook up custom stdio streams - one gets all the benefits of libc's buffering and formatting and one only has to write a couple of simple callbacks.

#define _GNU_SOURCE

#include <sys/types.h>
#include <sys/uio.h>
#include <stdio.h>
#include <unistd.h>

static ssize_t fcgi_write(void *f, const char *buf, size_t size, int type) {
        struct fcgi *cgi = f;
        size_t sent = 0;
        FCGI_Header header = {
                .version = FCGI_VERSION_1,
                .type = type,
                .requestIdB1 = cgi->rid1,
                .requestIdB0 = cgi->rid0
        };

        while (sent < size) {
                size_t left = size - sent;
                ssize_t res;
                struct iovec iov[2];

                if (left > 65535)
                        left = 65535;

                header.contentLengthB1 = left >> 8;
                header.contentLengthB0 = left & 0xff;

                iov[0].iov_base = &header;
                iov[0].iov_len = sizeof(header);
                iov[1].iov_base = (void *)(buf + sent);
                iov[1].iov_len = left;
                
                res = writev(cgi->fd, iov, 2);
                if (res < 0)
                        return -1;

                sent += left;
        }

        return size;
}

static int fcgi_close(void *f, int type) {
        struct fcgi *cgi = f;
        FCGI_Header header = {
                .version = FCGI_VERSION_1,
                .type = type,
                .requestIdB1 = cgi->rid1,
                .requestIdB0 = cgi->rid0
        };
        if (write(cgi->fd, &header, sizeof(header)) < 0)
                return -1;
        return 0;
}
  

Well perhaps the callbacks are more `straightforward' than simple in this case. FastCGI has a payload limit of 64K so any larger writes need to be broken up into parts. I use writev to write the header and content directly from the library buffer in a single system call (a pretty insignificant performance improvment in this case but one nonetheless). I might need to handle partial writes but this works so far - in which case the writev approach gets too complicated to bother with.

The actual 'cookie' callbacks just invoke the functions above with the FCGI channel to write to.

  
static ssize_t fcgi_stdout_write(void *f, const char *buf, size_t size) {
        return fcgi_write(f, buf, size, FCGI_STDOUT);
}

static int fcgi_stdout_close(void *f) {
        return fcgi_close(f, FCGI_STDOUT);
}

const static cookie_io_functions_t fcgi_stdout = {
        .read = NULL,
        .write = fcgi_stdout_write,
        .seek = NULL,
        .close = fcgi_stdout_close
};
  

And opening a custom stream is as as simple as opening a regular file.

static int fcgi_begin(struct fcgi *cgi) {
        cgi->stdout = fopencookie(cgi, "w", fcgi_stdout);

        ...;

        return 0;
}
  

Example

Here's a basic example that just dumps all the parameters to the client. It also maintains a count to demonstrate that it's persistent.

I went with a callback mechanism rather than the polling mechanism of the original SDK mostly to simplify managing state. Shrug.

#include "fcgi.h"

static int cgi_func(struct fcgi *cgi, void *data) {
        static int count;

        fprintf(cgi->stdout, "Content-Type: text/plain\n\n");
        fprintf(cgi->stdout, "Request %d\n", count++);
        fprintf(cgi->stdout, "Parameters\n");
        for (int i=0;i<cgi->param_length;i++)
                fprintf(cgi->stdout, " %s=%s\n", cgi->param[i].name, cgi->param[i].value);

        return 0;
}

int main(int argc, char **argv) {
        struct fcgi * cgi = fcgi_alloc();
        
        fcgi_accept_all(cgi, cgi_func, NULL);
        
        fcgi_free(cgi);
}

Notes

I haven't worked out how to get the CGI script to 'exit' when the MaxRequestsPerProcess limit has been reached without causing service pauses. Whether I do nothing or whether I exit and close the socket at the right time it still pauses the next request for 1-4 seconds.

I haven't converted my blog driver to use it yet - maybe later on tonight if I keep poking at it.

Oh and it is quite fast, even with a trivial C program.

Tagged hacking, zedzone.
Tuesday, 19 December 2017, 20:26

jjmpeg, jni, javafx

So I guess the mood took me, I somehow ended poking away until the very late morning hours (4am) the last couple of nights hacking on jjmpeg. Just one more small problem to solve ... that never ended. Today I should've been working but i've given up and will write it off, it's nearly xmas break anyway so there's no rush, and i'm ahead of the curve anyway.

JJMediaReader

I got this ported over and playing video fairly easily, and then went through on a cleanup spree. I removed all the BufferedImage, multi-buffering, and scaling stuff and a few other experiments which never worked. Some api changes allowed me to consolidate more code into a base class, and some changes to AVStream necessitated a different approach to initialising the AVCodecContext (using AVCodecParameters). I made a few other little tweaks on the way.

The reason I removed the BufferedImage code is because I didn't want to pollute it with "platform specific" code. i.e. swing, javafx, etc. I've moved that functionality into a separate namespace (module?).

My first cut just took the BufferedImage code and put it into another class which provides the functionality by taking the current AVFrame from the JJMediaReader video stream. This'll probably do but when working on similar functionality for JavaFX I took a completely different approach - implementing a native PixelReader() so that the native code can decide the best way to write to the buffer. This is perhaps a little more work but is a lot cleaner to use.

swscale

jjmpeg1 lets you scale images 'directly' to/from primitive arrays or direct ByteBuffers in addition to AVFrame. Since they have no structure description (size, format), this either has to be passed in to the functions (messy) or stored in the object (also messy). jjmpeg1 used the latter option and for now I simply haven't implemented them.

The PixelReader mentioned above does implement it internally but for code re-use it might make sense to implement them with the structure information as explicit parameters, and use higher level objects such as PixelReader/Writer to track such information. On the other hand the native code has access to more information so it also makes sense to leave it there.

I went a bit further and created a re-usable super-class that does most of the work and toolkit specific routines only have to tweak the invocation. This approach hides libswscale behind another api. The slice conversions don't work properly but they're not necessary.

jni

So far I had public constructors and `finalisers' because otherwise the reflection code failed. That's a bit too ugly (and `dangerous') so I made them private. The reflection code just had to look up the methods and set them Accessible.

    Constructor cc = jtype.getDeclaredConstructor(Long.TYPE);

    cc.setAccessible(true);

    return cc.newInstance(p);

Whilst working on JJMediaReader I hit a snag with the issue of ownership. In most cases objects are either created anew and released (or gc'd) by the Java code, or are simply references to data managed elsewhere. I was addressing the latter problem by simply having an empty release() method for the instance, but that isn't flexible enough because some objects are created or referenced the the context determines which.

So I expanded the Java-side object tracking to include a `refer' method in addition to the 'resolve' method. `resolve' either creates a new instance or returns and existing one with a weak-reference object which will invoke the static release method when it gets finalised. `refer' on the other hand does the same thing but uses a different weak-reference object which does nothing.

I then noticed (the rather obvious) that if an object is created, it can't possibly 'go away' from the object tracking if it is still alive; therefore the `resolve' method was doing redundant work. So I created another `create' method which assumes the object is always a new one and simply adds it to the table. It can also do some checking but i'm pretty sure it can't fail ...

If on the other hand the underlying data was reference counted then the `resolve' method would be useful since it would be possible to lookup an existing object despite it being `released'. So i'll keep it in CObject.

As part of this change I also improved CObject in other ways.

I was storing the weak reference to the object itself inside the object so I could implement explicit release and to avoid copying the pointer. I removed that reference and only store the pointer now. The WeakReference it already tracked in a hash table so I just look it up if I need it. This lets me change the jni code to use a field lookup rather than a function call to retrieve it (I doubt it makes much perf difference but I will profile it at some point).

I also had some pretty messy "cross-layer" use of static variables and messy synchronisation code. I moved all map references to outside of the weak reference routine and use a synchronised map for the pointer to object table.

For explicit release I simply call .clear() and .enqueue() on the WeakReference - which seems to do the right thing, and simplifies the release code (at least conceptually) since it always runs on the same thread.

Tagged hacking, java, javafx, jjmpeg.
Sunday, 05 November 2017, 14:51

JNI and garbage collection

I've started on an article about creating garbage collectible JNI objects. This is based on the system used in zcl but simplified further for reuse by using the class object as the type specifier and binding release via static declared methods.

This also supports `safe' explicit release which may be required in some circumstances where the gc is not run often enough.

It should work well with the JVM as it uses reference queues and no finalize methods. It requires minimal "extra" application support - just a class specific release() method.

Read it here.

Tagged code, hacking, java.
Newer Posts | Older Posts
Copyright (C) 2018 Michael Zucchi, All Rights Reserved.Powered by gcc & me!