archive snowblossom slack dev
prev 2019-05-30 next
2019-05-30 16:03:29
Rotonen
what's that for?
2019-05-30 16:04:13
Fireduck
So my theory that part of the problem I am seeing is thrashing things in and out of L3 cache
2019-05-30 16:04:24
Fireduck
so this allows only one wave to do the main processing at a time
2019-05-30 16:04:43
Fireduck
but allows you to still have other waves so that the reads from storage are done ahead of time
2019-05-30 16:04:53
Rotonen
you're reading at max speed at all the times, that'll just roll through the L3 in any case?
2019-05-30 16:05:17
Fireduck
current processor has 64mb L3, I am doing 32mb blocks
2019-05-30 16:05:28
Fireduck
not sure if that will help or not
2019-05-30 16:05:44
Rotonen
i'd read in 1G chunks as that gives the operating system and hardware more to do on their own
2019-05-30 16:05:50
Fireduck
I am kinda trying to reconstruct what is actually happening based mostly on how I imagine things work
2019-05-30 16:06:11
Rotonen
one thing is that the controller of the disk has an internal queuing fight
2019-05-30 16:06:16
Rotonen
vs. having just one clean long read in it
2019-05-30 16:06:45
Fireduck
yeah, that makes sense. I'd have to do a bigger redesign to read 1gb chunks and then break those up into cache size blocks
2019-05-30 16:06:50
Fireduck
not hard, just work
2019-05-30 16:06:51
Rotonen
for throughput, you pretty much want to minimize queue depth
2019-05-30 16:06:59
Rotonen
for random reads you want to make sure you're always having a deep queue
2019-05-30 16:07:32
Fireduck
this run with the cache lock is looking really good
2019-05-30 16:07:44
Fireduck
maxing out the cpu nicely, reading at 2GB/s
2019-05-30 16:07:52
Rotonen
i guess you just actually do less reads per second
2019-05-30 16:09:48
Rotonen
did `MappedByteBuffer` go away in java 8?
2019-05-30 16:11:24
Fireduck
in some testing I did for my electrum server, I found that it doesn't matter if you are doing memory mapped buffers vs file reads
2019-05-30 16:11:33
Fireduck
at least in Java
2019-05-30 16:11:47
Rotonen
i mean it'd help you with removing the layer you'd need to build for 1G reads?
2019-05-30 16:15:24
Fireduck
maybe. I'm not convinced anything matters with nvme doing 32mb reads vs 1gb reads. The slow part is the hashing on the cpu and thrashing memory around, I think
2019-05-30 16:15:38
Fireduck
maybe my MagicQueue is slowing things down
2019-05-30 16:15:52
Rotonen
that starts sounding like a threadripper should do well
2019-05-30 16:16:06
Rotonen
4 channels of ram, helluva lot of pcie, many cores
2019-05-30 16:16:06
Fireduck
that is what I am testing on right now
2019-05-30 16:16:27
Fireduck
AMD Ryzen Threadripper 1950X 16-Core Processor
2019-05-30 16:17:12
Fireduck
The number of pins on that socket was somewhat frightening
2019-05-30 16:17:29
Rotonen
that's actually a power delivery restriction
2019-05-30 16:17:36
Fireduck
also, like an idiot I bought a heat sink as big as my head
2019-05-30 16:17:47
Fireduck
which was nearly impossible to install
2019-05-30 16:17:49
Rotonen
you do the math, but it runs on 1 volt and burns over 100W worth of electricity :stuck_out_tongue:
2019-05-30 16:17:59
Fireduck
since the motherboard has a big heat sink blocking where i needed to get in
2019-05-30 16:18:13
Fireduck
that would be 100A, lol
2019-05-30 16:18:14
Fireduck
good
2019-05-30 16:18:19
Rotonen
it also actually is
2019-05-30 16:18:26
Rotonen
that's why you need many small pins
2019-05-30 16:18:52
Rotonen
designing motherboards is not the most trivial thing to do
2019-05-30 16:22:54
Fireduck
Yeah. 3d factorio
2019-05-30 16:23:07
Fireduck
Plus interfere
2019-05-30 16:23:20
Rotonen
plus thermodynamics
2019-05-30 18:15:11
Rotonen
did you try to style up the site? :smile:
2019-05-30 18:15:41
Rotonen
now it's a bit silly looking on a wide screen
2019-05-30 18:27:07
Fireduck
I noticed that but didn't do it intentionally
2019-05-30 18:40:12
Rotonen
have you considered using a static page generator?
2019-05-30 18:40:35
Fireduck
I am a static page generator. :wink:
2019-05-30 18:40:39
Fireduck
but not a good one
2019-05-30 18:40:44
Rotonen
something like hugo or sphinx where the inputs are markdown or restructured text
2019-05-30 18:43:39
Rotonen
i’m considering putting the wizard onto this https://forestry.io/ Static CMS for Hugo, Jekyll, VuePress and Gatsby sites, powered by Git. A better way to edit static sites.
2019-05-30 18:58:25
Fireduck
That is actually something I need for foreverbucket
2019-05-30 19:07:11
Rotonen
markdown covers what and how you want to tell something pretty well
2019-05-30 19:07:24
Rotonen
and someone else figured out how to shot web