the surfminer now reads only 1G/s for me whereas previously it read 3G/s

let’s see if it finally submits valid shares

at least it had already had trouble parsing the status message from the pool

4 waves, 16 hash threads, 40G work mem, 55G heap size

seems the GC stalls the waves

i’m now also convinced surf miner is cpu limited

and it’s not the gc, just that the queue is not progressing - the gc thread in iotop just coincides with the program being less busy

correlation does not equal causation

averaging 5 hashes per second over the first hour, but at least no invalid shares yet :P

@Fireduck how about you only log the share info line when a new share is found instead of every time the hashrate report fires?

so on a quick parametre sweep it’d seem the wave count should equal system hyperthreads and the hash thread count real cores

which makes sense, the 1G cost per wave can make that rough

you can change the wave size

The strange thing with burning all the cpu...PoolMiner in ram mode does all the fetching and gets a much higher hash rate, so my nonsense about the L3 cache size probably means nothing

something else is eating all the CPU

probably mallocs in the MagicQueue

what is a good java profiler? Ideally command line to gather data. Graphical to review the data would be fine.

last time i profiled java was on mobile java 1.4

you just need a call trace and a way to visualize

no fancy bells or whistles required

yep

is there a separate .proto for the pool communication?

yeah, mining_pool.proto

i’m actually gonna take a stab at putting together a prototype in pypy

excellent

it depends on the main proto though

i guess that’s streaming new blocks and pushing shares

for the block header

yep

fair

SharedMiningService = Arktika

i’ll start with the build tooling and other build system and packaging stuff

service MiningPoolService { rpc GetWork ( GetWorkRequest ) returns (stream WorkUnit) {} rpc SubmitWork ( WorkSubmitRequest ) returns ( SubmitReply ) {} }

That is the actual pool protocol, pretty simple

and submit reply is bool?

and work unit is int?

SubmitReply is a bool and an error message

or byte array, as every data type, and the game is a hunt for a prefix

The WorkUnit has a block header, a target hash and an identifier

i think i have a fair idea of the whole architecture i want to try

cool

how do you do the hashrate estimates without those getting in the way?

i think i’ll just estimate the 1h average based on share diff and count

What do you mean getting in the way?

and there is no inherent reason why one could not go for non-integer log2 difficulties?

well you need to keep books per crank of the handle

and that happens up to millions of times per second

MultiAtomicLong

so just a guaranteed locked int

https://github.com/fireduck64/duckutil/blob/master/src/MultiAtomicLong.java ``` package duckutil; import java.util.concurrent.atomic.AtomicLong; import java.util.LinkedList; /** * High performance atomiclong for multiple threads to write to quickly. * Does not do well if there are many threads being created as each one * will leave behind an entry. Works great with consistent thread pools. * * In testing on a reasonable multicpu machine with 100 threads, * got 60M/s with normal AtomicLong and 650M/s with this. */ public class MultiAtomicLong { private LinkedList<AtomicLong> al_list = new LinkedList<>(); private ThreadLocal<AtomicLong> al_local = new ThreadLocal<>(); /** * Get the sum of all values and reset to zero */ public long sumAndReset() { long v = 0; synchronized(al_list) { for(AtomicLong l : al_list) { v+= l.getAndSet(0L); } } return v; } public long sum() { long v = 0; synchronized(al_list) { for(AtomicLong l : al_list) { v+= l.get(); } } return v; } public void add(Long v) { AtomicLong l = al_local.get(); if (l == null) { l = new AtomicLong(0L); al_local.set(l); synchronized(al_list) { al_list.add(l); } } l.getAndAdd(v); } } ```

meh, i’ll avoid that for now

Way faster than an AtomicLong

well, python’s not for such

but share validation on the pool side, does it only do whole number log2 shares?

yeah

wanna eventually handle fractional prefixes as well?

I don't think so

doesn't seem like something we would need

I think the miners would support it currently

it’s just a bunch of bits, so yep

i predict i'll hit my head on the madness of python multiprocessing and shared memory and trying to do an LRU cache on that for the reads

or might just get stuck trying to find a way to run skein

to go the pypy way i'll need to reimplement stuff in python anyway

at least i have the worst parts of the lego i seek already done for me https://pypi.org/project/geesefly/ Pure Python implementation of Skein and Threefish

nice

or as that's half bitrot i'll probably try normal python for now as there is already https://pythonhosted.org/pyskein/

this project would have been impossible without the bouncycastle library

@Fireduck have you evaluated this one? https://github.com/bigfatbrowncat/zetes A lightweight cross-platform GUI application framework based on Avian and SWT

i have not

looks... stable

oh god, mingw