what's that for?
So my theory that part of the problem I am seeing is thrashing things in and out of L3 cache
so this allows only one wave to do the main processing at a time
but allows you to still have other waves so that the reads from storage are done ahead of time
you're reading at max speed at all the times, that'll just roll through the L3 in any case?
current processor has 64mb L3, I am doing 32mb blocks
not sure if that will help or not
i'd read in 1G chunks as that gives the operating system and hardware more to do on their own
I am kinda trying to reconstruct what is actually happening based mostly on how I imagine things work
one thing is that the controller of the disk has an internal queuing fight
vs. having just one clean long read in it
yeah, that makes sense. I'd have to do a bigger redesign to read 1gb chunks and then break those up into cache size blocks
not hard, just work
for throughput, you pretty much want to minimize queue depth
for random reads you want to make sure you're always having a deep queue
this run with the cache lock is looking really good
maxing out the cpu nicely, reading at 2GB/s
i guess you just actually do less reads per second
did `MappedByteBuffer` go away in java 8?
in some testing I did for my electrum server, I found that it doesn't matter if you are doing memory mapped buffers vs file reads
at least in Java
i mean it'd help you with removing the layer you'd need to build for 1G reads?
maybe. I'm not convinced anything matters with nvme doing 32mb reads vs 1gb reads. The slow part is the hashing on the cpu and thrashing memory around, I think
maybe my MagicQueue is slowing things down
that starts sounding like a threadripper should do well
4 channels of ram, helluva lot of pcie, many cores
that is what I am testing on right now
AMD Ryzen Threadripper 1950X 16-Core Processor
The number of pins on that socket was somewhat frightening
that's actually a power delivery restriction
also, like an idiot I bought a heat sink as big as my head
which was nearly impossible to install
you do the math, but it runs on 1 volt and burns over 100W worth of electricity :stuck_out_tongue:
since the motherboard has a big heat sink blocking where i needed to get in
that would be 100A, lol
good
it also actually is
that's why you need many small pins
designing motherboards is not the most trivial thing to do
Yeah. 3d factorio
Plus interfere
plus thermodynamics
did you try to style up the site? :smile:
now it's a bit silly looking on a wide screen
I noticed that but didn't do it intentionally
have you considered using a static page generator?
I am a static page generator. :wink:
but not a good one
something like hugo or sphinx where the inputs are markdown or restructured text
i’m considering putting the wizard onto this https://forestry.io/ Static CMS for Hugo, Jekyll, VuePress and Gatsby sites, powered by Git. A better way to edit static sites.
That is actually something I need for foreverbucket
markdown covers what and how you want to tell something pretty well
and someone else figured out how to shot web