| --- Log opened Fri May 23 00:00:10 2008 |
| 00:18 | <@McMartin> | Operation: Massive Cathartic Violence: Phase 1 complete |
| 00:21 | <@MyCatVerbs> | McM bought a copy of GTA4? |
| 00:24 | <@ToxicFrog> | No, he's reorganizing the UQM data. |
| 00:31 | <@MyCatVerbs> | Ahhh. Heh. Interesting name for the project. |
| 00:31 | | Shoukanjuu [~Shoukanju@Nightstar-19663.dhcp.embarqhsd.net] has joined #code |
| 00:31 | < Shoukanjuu> | I'm alive \o/ |
| 00:32 | <@MyCatVerbs> | Still alive! |
| 00:32 | | * MyCatVerbs gives Shoukanjuu a cookie for being not dead yet. |
| 00:38 | <@McMartin> | MCV: It's basically an SVN move on nearly every file in content. |
| 00:39 | <@McMartin> | All 12,000 of them. |
| 01:15 | <@MyCatVerbs> | McMartin: yikes. Did this require taking a pretty much exclusive hold on the repo for a little while? |
| 01:16 | <@McMartin> | No more than any other SVN commit |
| 01:16 | <@McMartin> | During the move operations and army of robots work and etc. I don't touch the network at all |
| 01:57 | <@MyCatVerbs> | Presumably if anyone else was hacking on things at the same time though, you'd get piles o' conflicts. |
| 01:57 | <@MyCatVerbs> | Or is SVN clever enough to shift the changes into the new place? |
| 01:59 | <@Kazriko> | on a move? |
| 02:00 | <@Kazriko> | I've never tried that before. :) |
| 02:01 | | * Kazriko ponders trying that. |
| 02:06 | <@ToxicFrog> | Well, does SVN implement mv as a primitive operation or as delete+add? |
| 02:09 | <@Kazriko> | it does support move, but as kind of a branch operation. |
| 02:09 | <@Kazriko> | it's a branch and delete |
| 02:09 | <@Kazriko> | let me try it out here. |
| 02:16 | <@Kazriko> | moving a single file, it doesn't automatically move your files over, but it doesn't delete your altered copy either. |
| 02:18 | <@Kazriko> | same for moving a directory. |
| 02:43 | <@McMartin> | The actual content has been stable for years |
| 02:56 | <@McMartin> | Aaaand, the commit of doom begins |
| 02:57 | <@McMartin> | There will then be three or four additional commits of noticably lesser doom. |
| 03:26 | | JPL [8036456e@Nightstar-9250.mibbit.com] has joined #code |
| 03:30 | < JPL> | Don't you just hate it when you're presented with a simple problem you know there's a simple answer to, but can't remember how to at all? |
| 03:39 | <@McMartin> | ... wow, SVN just shat itself but good |
| 03:40 | <@Vornicus> | ...ooops |
| 03:40 | <@Vornicus> | JPL: been there done that |
| 03:40 | <@McMartin> | WHen dealing with fonts it attempted to delete, then branch |
| 03:40 | <@McMartin> | And then errored out but let the commit go through anyway |
| 03:41 | <@Vornicus> | ...uh... |
| 03:41 | | * McMartin sets about repairing the damage |
| 03:56 | < JPL> | I have a ... pointer, to a ... list of pointers . The good news is I know how many pointers there are, the bad news is I can't recall for the life of me how to seperate them out into individual pointers. |
| 03:59 | <@Vornicus> | a pointer and an array are the same thing |
| 04:00 | < JPL> | Yes, I am aware of this. If NachOS was still written in C, I could pass in the top pointer in the stack and call it a day. Unfortunately, I have to unpack the pointers, read their locations from virtual memory, pack that into an array, and send it to the exec() call. |
| 04:00 | <@Vornicus> | what the shit? |
| 04:03 | < JPL> | Yup. |
| 04:08 | <@McMartin> | You know the size of the list. Allocate an array of that size, iterate through list, copy each element into array |
| 04:16 | < JPL> | I know. The problem is I've forgotten how to turn the array I have that's a list of pointers into individual pointers. I might be able to do this just by reading the right bytes, if 1) I rememebered how to do that in Java and 2) I knew how long the pointer was (4 bytes?). |
| 04:17 | <@McMartin> | Doesn't your list node class have a getContents() method or something? |
| 04:18 | < JPL> | list node class? |
| 04:18 | <@McMartin> | You've got a list of whatevers, right? |
| 04:21 | < JPL> | List of pointers... represented as a string, all jammed together. |
| 04:21 | < JPL> | It's a system call. |
| 04:21 | < JPL> | Very low level. |
| 04:23 | <@McMartin> | Uh |
| 04:23 | <@McMartin> | You're writing in Java |
| 04:23 | <@McMartin> | Java doesn't have system calls |
| 04:24 | <@McMartin> | Not directly exposed |
| 04:24 | <@McMartin> | There are at least three different kinds of heinousness they could be unleashing on you here. |
| 04:25 | <@McMartin> | But I *suspect* what you want is charAt(), combined with some kind of building the integer out of bytes. |
| 04:25 | < JPL> | I know. This is trying to implement a system call in NachOS |
| 04:29 | <@McMartin> | OK |
| 04:29 | <@McMartin> | You're full of level confusion |
| 04:29 | <@McMartin> | You are writing, in Java, a simulator of a low level system. |
| 04:29 | <@McMartin> | The "pointers" you're dealing with, if they're Objects, are pointers to stuff *in the simulator*. |
| 04:29 | <@McMartin> | The stuff you're trying to unpack are pointers *in the simulated system* |
| 04:29 | <@McMartin> | If I understand your description right. |
| 04:30 | <@McMartin> | The pointers are almost certainly 32 bits long |
| 04:30 | <@McMartin> | It is plausible that the string encoding is actually taking each 8-bit value, taking a UTF-8 encoding of it, and making a string out of the result |
| 04:31 | <@McMartin> | This means that Java's charAt() will give you a value as if the String were a byte array, which is what it should have frickin' been in the first place |
| 04:31 | < JPL> | Yes. |
| 04:32 | <@McMartin> | In order to turn an array of octets into an array of pointer values, you need (1) the width of a pointer (almost certainly 4 bytes) and (2) the endianness of the simulated system. |
| 04:32 | <@McMartin> | And then you perform computations involving bit shifts and masks and stuff and end up with an array of ints. |
| 04:32 | | * Vornicus pokes vaguely at his computer, wonders if he can watch a program pick up fds, and from where... |
| 04:37 | < JPL> | Let's say I had this in a byte[] array. What would I need to do to chop the first pointer off it, in java? |
| 04:38 | <@McMartin> | The depends on whether it's big-endian or little endian |
| 04:38 | <@McMartin> | If it's big-endian, it would be something like |
| 04:39 | <@McMartin> | int x = 0; for (int i = 0; i < 4; i++) { x <<= 8; x |= a[i]; } |
| 04:40 | <@McMartin> | You read out each byte one at a time and shift it to its location in the result |
| 04:40 | <@McMartin> | Big-endian, that's 0 1 2 3 4 |
| 04:40 | <@McMartin> | Little-endian, that's 4 3 2 1 |
| 04:40 | <@McMartin> | So, 0xDEADBEEF would encode to a byte array of 0xEF, 0xBE, 0xAD, 0xDE |
| 04:41 | <@McMartin> | Bigendian, that's 0xDE, 0xAD, 0xBE, 0xEF |
| 04:41 | < JPL> | Ok. |
| 04:43 | <@McMartin> | The shift left operator: x << 8 turns 0xDE into 0xDE00. |
| 04:43 | < JPL> | Right. |
| 04:43 | <@McMartin> | And logical-or copies in a bunch of 1 bits |
| 05:01 | | JPL [8036456e@Nightstar-9250.mibbit.com] has quit [Quit: http://www.mibbit.com ajax IRC Client] |
| 05:02 | | JPL [8036456e@Nightstar-9250.mibbit.com] has joined #code |
| 05:58 | | JPL [8036456e@Nightstar-9250.mibbit.com] has quit [Quit: http://www.mibbit.com ajax IRC Client] |
| 06:30 | | AnnoDomini [AnnoDomini@Nightstar-6998.neoplus.adsl.tpnet.pl] has joined #Code |
| 06:30 | | mode/#code [+o AnnoDomini] by ChanServ |
| 06:33 | <@Vornicus> | Okay, tell me if I fell off the bus here, but. |
| 06:35 | <@Vornicus> | the Bittorrent Patch Device: when a patch comes out, you download a new .torrent and shake it out, so it checks all the chunks and downloads only the ones you need. |
| 06:36 | <@Vornicus> | Advantages: you don't need to jump through multiple patches to get to the current version, no matter what version you're on now; you don't need to download entire files if only parts of files change (with some restrictions)... |
| 06:42 | | Thaqui [~Thaqui@Nightstar-711.jetstream.xtra.co.nz] has joined #code |
| 06:42 | | mode/#code [+o Thaqui] by ChanServ |
| 06:43 | <@Vornicus> | in fact, I can't think of any negatives |
| 06:44 | <@Vornicus> | I guess it's a bit less efficient download-size-wise than keeping a complete pile of .patches, but |
| 06:49 | <@Vornicus> | that seems more than made up for by the speed and infrastructure benefits of using a shotgun. |
| 07:04 | <@Vornicus> | A shotgun that shoots shotguns even |
| 07:09 | < Shoukanjuu> | Might I suggest A shotgun that shoots shotguns that shoots exploding chainsaws? To the max! |
| 07:09 | | Shoukanjuu is now known as Shou|x_X |
| 07:10 | <@Vornicus> | awesome |
| 07:10 | <@Kazriko> | Vornicus, i think the problem might be that bittorrent wasn't designed to alter existing files, only get more pieces of not yet existing files? |
| 07:10 | <@Kazriko> | if you could do something with a bittorrented rsync. |
| 07:10 | <@Kazriko> | i think bittorrent would end up throwing out entire files even if only a little of them changed and the rest shifted? |
| 07:11 | | * Kazriko wonders if this is somewhere that a git style thing could come into play, with a peer to peer sort of thing added in ala bittorrent. |
| 07:12 | <@Vornicus> | Kazriko: this is what checksums are for. Bittorrent is perfectly capable of altering existing files, so long as their length doesn't change. It will refetch the beginning of a file if the end of the previous gets mangled, and vice versa. |
| 07:12 | <@Kazriko> | that's the thing. the length would likely change on an upgrade. |
| 07:12 | <@Vornicus> | the trouble with shifting is a bit of a problem, really, but I've never met a syncing thing that knows how to do that right; it will just replace. |
| 07:12 | <@Kazriko> | other than having to completely reget altered files, it's probably a workable idea. |
| 07:13 | <@Kazriko> | I think Sony's done roughly the same thing with their inu protocol. |
| 07:13 | <@Kazriko> | it's a bittorrent style updater for files in games. |
| 07:14 | <@Vornicus> | A good directory structure would be Useful to say the least |
| 07:14 | <@Vornicus> | And you'll probably want to use Append more than Insert |
| 07:15 | <@Vornicus> | (because appending doesn't shift anything) |
| 07:16 | <@Kazriko> | as long as you don't have really big files ala doom's wad format and such. |
| 07:17 | <@Vornicus> | Right. |
| 07:17 | <@Vornicus> | Zips and wads don't really work that well for this kind of work. |
| 07:19 | <@Kazriko> | man, nobody has any info at all on Sony Inu |
| 07:19 | <@Kazriko> | I guess they're not talking about it. I only know about it because I saw debug logs for one of the games. |
| 07:20 | <@Vornicus> | you'd need FUSE-like functionality to get into those and make that efficient |
| 07:20 | <@Kazriko> | yeah... |
| 07:21 | <@Vornicus> | And separate it very, very carefully |
| 07:21 | <@Vornicus> | Essentially all the patch-size-minimization stuff you can think of works well here |
| 07:22 | <@Kazriko> | Why I was thinking of git, was that it probably stores deltas, and you could look at the delta between your version and the master version to get an efficient patch for a shifted file like that... |
| 07:23 | <@Vornicus> | But that gets us back to the cumulative-patch problem that WoW etc has - you have to get the delta from 1.0 to 1.1 then to 1.3 then to 2.0 then to 2.4 |
| 07:24 | <@Kazriko> | when you do a git pull, does it do all the deltas in one shot? |
| 07:24 | <@Vornicus> | I don't know. |
| 07:24 | <@Kazriko> | meither. heh |
| 07:25 | <@Vornicus> | It probably constructs the delta live, which is either lots of I/O and processing or lots of caching. |
| 07:25 | <@Vornicus> | And either way you need to keep the old ones around and 'public' enough that you can run the deltas when needed. |
| 07:31 | <@Vornicus> | One of the big advantages of the torrent version is that you don't need to care, period, what they're upgrading from. |
| 07:32 | <@Kazriko> | nod. |
| 07:32 | <@Vornicus> | Somebody comes on with WoW 1.0 and a license key for Wrath of the Lich King? You need to do exactly as much work as someone who's coming on with Burning Crusade 2.3 |
| 07:33 | <@Vornicus> | Which is to say, give him a new .torrent, let your Army Of Robots handle the rest. |
| 07:35 | <@Vornicus> | Pre-alpha build? Taken care of, just give him the .torrent and have fun. Nothing at all? .torrent |
| 07:37 | | * Kazriko wonders how impulse does that. |
| 07:37 | <@Vornicus> | well. nothing at all is .torrent plus torrent client, but |
| 07:38 | <@Kazriko> | actually, that's also similar to how ffxi updates. |
| 07:38 | <@Kazriko> | it checks every file to make sure the checksum matches. |
| 07:38 | <@Kazriko> | and updates only those that differ |
| 07:43 | <@Vornicus> | Yah. Same thing here, except Bittorrent is more efficient on lightly-modded files. |
| 07:45 | <@Vornicus> | (lightly-modded being anything where there is change without significant shifting, in this case) |
| 07:47 | <@Vornicus> | well. and you can get others to contribute. |
| 07:50 | <@Vornicus> | I get the impression that WoW actually torrents a patch file and then applies it. |
| 07:50 | <@Kazriko> | nod. |
| 07:53 | <@Vornicus> | Which is the Wrong Way To Do It, really, because you have to checksum the files beforehand, checksum the patch, then checksum the files afterwards, and then go get the next patch... |
| 08:20 | | GeekSoldier [~Rob@Nightstar-10251.dip.t-dialin.net] has joined #code |
| 08:20 | | mode/#code [+o GeekSoldier] by ChanServ |
| 09:15 | | Attilla [~The.Attil@194.72.70.ns-11849] has quit [Quit: <Insert Humorous and/or serious exit message here>] |
| 09:34 | | Attilla [~The.Attil@194.72.70.ns-11849] has joined #code |
| 09:34 | | mode/#code [+o Attilla] by ChanServ |
| 10:10 | | Vornicus is now known as Vornicus-Latens |
| 10:28 | | Thaqui [~Thaqui@Nightstar-711.jetstream.xtra.co.nz] has left #code [Leaving] |
| 11:00 | | Mojo1978 [~Mojo1978@Nightstar-26104.hsi.ish.de] has joined #code |
| 13:19 | | AnnoDomini [AnnoDomini@Nightstar-6998.neoplus.adsl.tpnet.pl] has quit [Ping Timeout] |
| 13:26 | | AnnoDomini [AnnoDomini@Nightstar-29387.neoplus.adsl.tpnet.pl] has joined #Code |
| 13:26 | | mode/#code [+o AnnoDomini] by ChanServ |
| 15:07 | < AFKSkull> | vorn- wow does nearly EVERYTHING wrong on the back end. |
| 15:09 | < AFKSkull> | and especially when it comes to patching |
| 15:11 | < AFKSkull> | also, blizzard doesn't know the meaning of 'efficient'. |
| 15:11 | <@ToxicFrog> | Re: bittorrent operating on modified files: AFAIK this is an implementation issue, and all the clients I know of will handle non-shifting modifications properly (they'll only download the blocks that changed). Shifting of course means that all the blocks after the change are different, but even that doesn't require re-downloading the entire file unless the change was right at the start. |
| 15:12 | <@ToxicFrog> | If you know of any clients that handle changes to individual blocks by marking the whole file invalid, you should probably file bug reports against them :P |
| 15:14 | <@ToxicFrog> | Re: git-fetch: it can't do the deltas in one shot, generally, because fetch doesn't just get you the latest rev, it gets you all the revs you're missing - each repo has a complete copy of the history. |
| 15:14 | <@ToxicFrog> | Or, rather |
| 15:14 | <@ToxicFrog> | It can't generate one huge delta that takes you from where you are now to where the remote is, unless there are no intervening commits. |
| 15:18 | <@ToxicFrog> | In practice I'm not sure how it's implemented. |
| 17:09 | | You're now known as TheWatcher[afk] |
| 18:05 | | AnnoDomini is now known as Lance |
| 18:29 | | You're now known as TheWatcher |
| 19:26 | | GeekSoldier [~Rob@Nightstar-10251.dip.t-dialin.net] has quit [Ping Timeout] |
| 19:28 | | GeekSoldier [~Rob@Nightstar-8553.dip.t-dialin.net] has joined #code |
| 19:28 | | mode/#code [+o GeekSoldier] by ChanServ |
| 19:31 | <@MyCatVerbs> | ToxicFrog: makes me ponder whether a distributed rsync would be worthwhile. Or... extensions hacked on to Bittorrent that'd make it just about manage the same job that rsync does. |
| 19:36 | <@ToxicFrog> | struct.write() is working! |
| 19:37 | <@ToxicFrog> | MyCatVerbs: extend bittorrent to support file metadata and you're mostly done, I think. |
| 19:37 | <@ToxicFrog> | Well, file metadata and online update of .torrent info |
| 19:39 | <@MyCatVerbs> | Support for efficient insertions into the middles of files would be handy too. |
| 19:39 | <@ToxicFrog> | That's up to the filesystem. |
| 19:41 | <@Vornicus-Latens> | Though, now that I think about it |
| 19:42 | <@Vornicus-Latens> | I don't know any I/O libs that know about "insert into middle" |
| 19:44 | <@Vornicus-Latens> | So it's not like you're going to be outdoing that any time soon |
| 19:46 | <@ToxicFrog> | commmit'd |
| 19:47 | <@ToxicFrog> | Along with my first git "whoops forgot to pull earlier" merge |
| 19:47 | <@Vornicus-Latens> | Spelling, on the other hand, handles inserts in the middle quite well... |
| 19:47 | <@ToxicFrog> | ...heh. So it would appear. |
| 20:03 | <@Vornicus-Latens> | So, changes in files: changes at the beginning of one file (or the insertion of a new file) force the redownload of the end of the previous, and vice versa. Insertions or removals of parts of a file force the redownload of the parts of the file at and after the insertion. |
| 20:30 | | paarzh [~paarzh@Nightstar-11274.0-85.cust.bluewin.ch] has joined #code |
| 20:31 | | paarzh [~paarzh@Nightstar-11274.0-85.cust.bluewin.ch] has quit [Quit: http://irc.netsplit.de/] |
| 20:52 | | Vornicus-Latens is now known as Vornicus |
| 20:59 | | Serah [~Z@87.72.35.ns-26506] has quit [Quit: Don't try to read the quit message, that is impossible. Instead only realize the truth; "there is no quit message" and you will see it is not you who read the quit message but the quit message who reads you.] |
| 21:00 | <@ToxicFrog> | I need a name for the submodule that turns format strings into executable cocxde |
| 21:00 | <@ToxicFrog> | *code |
| 21:01 | <@Lance> | strToCode? |
| 21:01 | <@Vornicus> | "compile" |
| 21:02 | <@ToxicFrog> | I was thinking struct.compiler, yes |
| 21:03 | <@ToxicFrog> | Keffo suggests struct.executableizer |
| 21:03 | <@Vornicus> | that's just strange |
| 21:04 | <@Vornicus> | Python's regex module uses "compile" to create regex objects |
| 21:04 | | Raif [~corvusign@Admin.Nightstar.Net] has quit [Killed (NickServ (GHOST command used by Raif_))] |
| 21:04 | | Raif_ [~corvusign@Nightstar-25074.hsd1.wa.comcast.net] has joined #Code |
| 21:04 | | Raif_ is now known as Raif |
| 21:19 | | Mojo1978 [~Mojo1978@Nightstar-26104.hsi.ish.de] has quit [Ping Timeout] |
| 21:23 | | You're now known as TheWatcher[afk] |
| 21:33 | | Lance is now known as AnnoDomini |
| 21:35 | <@McMartin> | compile |
| 21:36 | <@McMartin> | If not "compile", then "translate" |
| 22:26 | | Serah [~Z@Nightstar-5401.atm2-0-1041217.0xc329e232.boanxx12.customer.tele.dk] has joined #Code |
| 22:26 | | mode/#code [+o Serah] by ChanServ |
| 22:26 | <@MyCatVerbs> | ToxicFrog: I was thinking efficient in terms of network traffic, rather than disk throughput, for insertions. |
| 22:28 | | You're now known as TheWatcher |
| 22:28 | <@MyCatVerbs> | ToxicFrog: 'specially since the reads 'n' writes you'd need to do in order to do the insertions naively would be almost completely sequential anyway, so you get beautiful speed anyway. |
| 22:28 | <@MyCatVerbs> | 'lo Serah, how's you? |
| 22:28 | <@Serah> | Five by five. |
| 22:28 | <@Serah> | &wr0kking. |
| 22:29 | <@MyCatVerbs> | Hrmn. I'm not sure how to interpret "loud and clear" on the emotional scale. |
| 22:29 | <@Serah> | today I will be playing Arcanum @ wr0k |
| 22:29 | <@MyCatVerbs> | I mean, are we talking merely normal "awesome" or in fact "so awesome that I'm wearing a Hawaaian shirt"? |
| 22:29 | <@Serah> | Everything is in perfect order. |
| 22:29 | <@AnnoDomini> | Night shift, Serah? |
| 22:30 | <@Serah> | Five by, refers to signal strength and clarity. |
| 22:30 | <@Serah> | Yup. |
| 22:30 | | * MyCatVerbs knwos. |
| 22:30 | <@Serah> | So everything in order. |
| 22:30 | <@MyCatVerbs> | It's just I don't knwo how you map strength and clarity onto emotional status. |
| 22:30 | <@MyCatVerbs> | I mean, literally loud and clear? |
| 22:31 | <@MyCatVerbs> | So much so as to require floral-print jumpers and gigantic mangoes rolling around in the hallways? |
| 22:31 | <@Serah> | Noone ever specified emotions, bar you. |
| 22:32 | <@MyCatVerbs> | Well, emotions and physical up. That's what I'm usually after when I ask "how's you?". |
| 22:32 | <@Serah> | I get paid to eat their food and drink their soda, how could it be any better? :p |
| 22:32 | <@Serah> | Oh. |
| 22:32 | <@MyCatVerbs> | I... I thought we had a convention. *sniff* :'( |
| 22:32 | <@Serah> | Well, in conflict about wether I want to take more shifts. |
| 22:32 | <@Serah> | I was offered all weekdays week 24-26. |
| 22:33 | | * MyCatVerbs starts wearing eyeliner and black clothing, buys a semi-broken guitar and starts improvising crap emo poetry in a tinny, nasal voice. |
| 22:33 | <@Serah> | I want the money, but I don't want all the hassle, and I won't be seeing my friends. |
| 22:34 | <@MyCatVerbs> | Hrmn. Take the money and run with it, and organize a low-key party with friends to make up for it. |
| 22:36 | <@Serah> | If I took the money I'd buy myself a new laptop. |
| 22:39 | <@Serah> | This compaq is ooold. |
| 23:26 | | AnnoDomini [AnnoDomini@Nightstar-29387.neoplus.adsl.tpnet.pl] has quit [Quit: Tradition may be defined as an extension of the franchise. Tradition means giving votes to the most obscure of all classes, our ancestors. It is the democracy of the dead. Tradition refuses to submit to the small and arrogant oligarchy of those who merely happen to be walking about. All democrats object to men being disqualified by the accident of birth; tradition objects to their being disqualified by the accident of death.] |
| --- Log closed Sat May 24 00:00:16 2008 |