archive snowblossom slack mining
prev 2018-12-23 next
2018-12-23 00:47:47
Fireduck
yeah, that should be 90k or so
2018-12-23 00:47:50
Fireduck
running enough threads?
2018-12-23 00:47:58
Fireduck
I'd expect to need 200 or 300
2018-12-23 01:27:26
Fable
I was getting that on my 3xx motherboard
2018-12-23 01:27:37
Fable
but not on the z270
2018-12-23 01:28:14
Fireduck
270 is less than 300
2018-12-23 01:28:32
Fable
Wait, what?
2018-12-23 01:28:44
Fable
I am talking about motherboard series, not threads
2018-12-23 01:29:49
Fable
where do you put nolisten=true for arktika to prevent port binding
2018-12-23 01:30:52
Fable
It doesn't do anything when you put it in the miner.conf
2018-12-23 01:31:17
Fireduck
A dumb joke
2018-12-23 01:31:32
Fireduck
One sec, it might be called something else
2018-12-23 01:32:52
Fireduck
nolisten=true in the arktika config file
2018-12-23 01:33:28
Fable
not preventing port binding
2018-12-23 01:33:33
Fable
when I do that
2018-12-23 01:34:13
Fable
or at least the second instance wont run
2018-12-23 01:34:16
Fable
sorry
2018-12-23 01:34:48
Fireduck
what error do you see?
2018-12-23 01:34:53
Fable
hold on
2018-12-23 01:35:11
Fireduck
and what version?
2018-12-23 01:38:51
Fable
1.3.3
2018-12-23 01:39:35
Fable
Exception in thread "main" java.io.IOException: Failed to bind at io.grpc.netty.NettyServer.start(NettyServer.java:252) at io.grpc.internal.ServerImpl.start(ServerImpl.java:162) at io.grpc.internal.ServerImpl.start(ServerImpl.java:77) at snowblossom.miner.Arktika.<init>(Arktika.java:147) at snowblossom.miner.Arktika.main(Arktika.java:65) Caused by: java.net.BindException: Address already in use: bind at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Unknown Source) at sun.nio.ch.Net.bind(Unknown Source) at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source) at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128) at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:558) at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1358) at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501) at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486) at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:1019) at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:254) at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:366) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:465) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Unknown Source)
2018-12-23 01:41:40
Fireduck
1.3.3? That was ages ago
2018-12-23 01:41:53
Fireduck
no listen is only in newer ones
2018-12-23 01:42:34
Fable
Do I have to bazel build it or is there a link
2018-12-23 01:44:44
Fireduck
updating that now
2018-12-23 01:44:53
Fireduck
if it is older than 1.4.2, wait a minute and download again
2018-12-23 01:55:32
Fireduck
The new version prints nice IO use messages
2018-12-23 01:56:21
Fable
still only get 18k on my z270
2018-12-23 01:56:31
Fable
cpu benches 300k
2018-12-23 03:20:04
Fable
Okay, this is going to be too much for me
2018-12-23 03:20:20
Fable
I now realize that you need a server setup to do this
2018-12-23 03:20:58
Fable
Can't work without xeons
2018-12-23 03:21:33
Fable
The cpu load is too high for a simple rig setup
2018-12-23 03:21:34
Fireduck
I've had to move m.2 drives around
2018-12-23 03:21:48
Fireduck
mostly I've have good luck with modern boards with m.2 on board over pcie adapters
2018-12-23 03:21:52
Fable
well when you run multiple instances of the miner you cant hit the single instance benchmark speed
2018-12-23 03:22:06
Fireduck
why would you run mujltiple instances?
2018-12-23 03:22:11
Fable
and the other instances start to slow
2018-12-23 03:22:17
Fable
well you wouldnt if you didnt have to
2018-12-23 03:22:40
Fable
So how would you run multiple nvmes on one arktika instance?
2018-12-23 03:22:48
Fireduck
what I mean is that with arktika, you should be able to configure a single one to use whatever resources you have
2018-12-23 03:22:57
Fireduck
yeah, split up the chunk files and put some on each nvme
2018-12-23 03:23:02
Fireduck
one layer for each nvme
2018-12-23 03:23:25
Fireduck
that should work well, but I admit I haven't tried that personally
2018-12-23 03:23:42
Fable
I didnt think that would work I thought that was for when the field started to get too large
2018-12-23 03:23:52
Fable
but now, yes, thanks
2018-12-23 03:24:07
Fireduck
it is also for distributing the load among block devices
2018-12-23 03:24:43
Fable
block devices?
2018-12-23 03:24:54
Fireduck
in this case the nvmes
2018-12-23 03:25:01
Fireduck
but could be ssd
2018-12-23 03:25:04
Fable
snowplows?
2018-12-23 03:25:05
Fireduck
or remote ram servers
2018-12-23 03:25:14
Fable
dumb joke
2018-12-23 03:25:19
Fireduck
ha
2018-12-23 03:26:58
Fable
I also thought you said that you couldn't use range parameter with layertype=file
2018-12-23 03:27:12
Fable
so that led to my assumption
2018-12-23 03:30:02
Fireduck
yeah, no range
2018-12-23 03:30:06
Fireduck
it uses whatever files are there
2018-12-23 03:30:10
Fireduck
so you have to move things around
2018-12-23 03:30:18
Fable
how would splitting up the chunks give you the full speed for each device running one miner isntance?
2018-12-23 03:30:20
Fireduck
I plan on adding the range in though, it will make it easier to tune
2018-12-23 03:30:29
Fable
yeah okay
2018-12-23 03:31:21
Fable
I guess, so you are renting resources to do this then?
2018-12-23 03:31:28
Fable
and just testing small samples
2018-12-23 03:36:26
Fireduck
I have a small memory mining fleet
2018-12-23 03:36:57
Fireduck
It is a single Dell r900 with 256gb ram and network workers
2018-12-23 03:37:06
Fireduck
Total about 1mh
2018-12-23 03:37:34
Fable
how would you configure that
2018-12-23 03:37:38
Fable
with the snow files
2018-12-23 03:37:51
Fable
just two snowfields?
2018-12-23 03:40:58
Fireduck
A single field 7
2018-12-23 03:41:23
Fireduck
More network limited than memory bus
2018-12-23 03:41:45
Fable
So it was registered ram
2018-12-23 03:43:55
Fable
splitting the chunks up still only gives you the speed of one 512 nvme?
2018-12-23 03:45:40
Fable
so yeah
2018-12-23 03:46:05
Fireduck
Should get combined speed
2018-12-23 03:46:06
Fable
not really viable unless the miner can cut cpu usage for same thread count
2018-12-23 03:46:27
Fireduck
What is the CPU?
2018-12-23 03:46:55
Fable
i9 7940x
2018-12-23 03:47:02
Fable
benches 860k
2018-12-23 03:47:12
Fireduck
Ok, cool
2018-12-23 03:47:51
Fable
just leave the whole snow field?
2018-12-23 03:47:57
Fireduck
How many nvme do you have?
2018-12-23 03:48:21
Fireduck
Put 32 chuncks on each
2018-12-23 03:49:00
Fable
im running field 8
2018-12-23 03:49:34
Fireduck
Have you seen the benchmark_layer option?
2018-12-23 03:49:39
Fable
fake
2018-12-23 03:50:01
Fireduck
Fake layer is a different thing
2018-12-23 03:50:09
Fireduck
Fake layer tests cpu
2018-12-23 03:50:13
Fable
okay
2018-12-23 03:50:16
Fable
then no
2018-12-23 03:51:40
Fireduck
benchmark_layer=0
2018-12-23 03:51:45
Fireduck
benchmark_threads=512
2018-12-23 03:51:57
Fireduck
that will just do as much IO on that layer as possible using that many threads
2018-12-23 03:52:04
Fireduck
that is a good way to see how much IO you can do on that layer
2018-12-23 03:52:24
Fireduck
so that would be good to see if your NVME can read as fast as spec says regardless of any CPU issues
2018-12-23 03:52:29
Fable
so delete other layers temporarily to test one
2018-12-23 03:52:33
Fable
in the config
2018-12-23 03:52:52
Fireduck
you can leave them in
2018-12-23 03:53:02
Fireduck
set benchmark_layer to the one you want to test
2018-12-23 03:55:27
Fable
read ops/s and read_bw
2018-12-23 03:55:36
Fable
582310 , 2275
2018-12-23 03:55:48
Fable
but says no hashing
2018-12-23 03:56:12
Fireduck
yeah, it is doing ont hashing
2018-12-23 03:56:18
Fireduck
no hashing
2018-12-23 03:56:22
Fireduck
just testing bandwidth
2018-12-23 03:56:34
Fable
I only did 1 benchmark layer though
2018-12-23 03:56:35
Fable
should I do 4
2018-12-23 04:01:09
Fireduck
yeah, it only supports one at time
2018-12-23 04:01:21
Fireduck
which is enough to let you know if the layer works as fast as you think it should
2018-12-23 04:01:51
Fable
if you can split the chunks up on a rig whats the point of 512gb drives
2018-12-23 04:02:16
Fable
I should be able to put the whole field on each drive and get the full hashrate with one miner instance, no?
2018-12-23 04:02:25
Fireduck
well, that will continue to work up to a much larger field size if split 4 ways
2018-12-23 04:02:41
Fable
still only 60k though
2018-12-23 04:02:59
Fireduck
if arktika supports that sort of load balancing, yes. You should absolutely be able to get nearly 400k with the rig you have
2018-12-23 04:03:17
Fireduck
assuming those nvmes are attached such that they can all be read at full rate at once
2018-12-23 04:03:22
Fable
so put the whole field on each drive?
2018-12-23 04:03:36
Fireduck
no, put 1/4th on each drive
2018-12-23 04:04:02
Fireduck
in some situations, you do need the data spread across the drive to get full rate, but I don't think we are there
2018-12-23 04:04:30
Fireduck
I think 64gb on each drive will be enough to spread it across whatever banks or chips or whatever there are
2018-12-23 04:05:03
Fireduck
So if you run four arktika instances you don't get good rates? (one per drive)
2018-12-23 04:05:18
Fireduck
can you show me one of the configs you were using for that?
2018-12-23 04:05:26
Fireduck
That should work as well
2018-12-23 04:08:30
Fable
the cpu load was too high
2018-12-23 04:08:45
Fable
my system has 128gb ram, I was running the bat with:
2018-12-23 04:08:59
Fable
@echo off title miner start /B /LOW java -jar -Xmx100g Arktika.jar configs/miner.conf pause
2018-12-23 04:09:21
Fable
and then 512 threads on each instance
2018-12-23 04:09:32
Fable
but the cpu load is way too high with that many threads
2018-12-23 04:09:39
Fable
but it takes that many threads to get 60k
2018-12-23 04:09:47
Fireduck
interesting
2018-12-23 04:09:47
Fable
otherwise I only get 20~k
2018-12-23 04:09:55
Fireduck
I haven't done much arktika testing on windows
2018-12-23 04:10:02
Fable
I don't know why the cpu load is so high though
2018-12-23 04:10:12
Fable
unless you are mining xmr
2018-12-23 04:10:13
Fireduck
but that -Xmx100g is way too high if you aren't trying to fit it in ram
2018-12-23 04:10:21
Fireduck
probably remove that entirely
2018-12-23 04:10:24
Fable
okay
2018-12-23 04:10:38
Fireduck
can you show me the miner.conf?
2018-12-23 04:11:01
Fable
mine_to_address=e82d6vzegevqdey6y05625hfsm87jgtrl5k85j42 network=snowblossom node_host=2338 pool_port=23380 memfield=false min_depth_to_disk=6 layer_count=1 layer_0_type=file layer_0_path=f:\\snow layer_0_threads=64 snow_path=f:\\Snow pool_host=http://findsnow.org selected_field=8 nolisten=true
2018-12-23 04:11:41
Fable
that was from before, except I changed the threads to lower cpu usage
2018-12-23 04:12:46
Fireduck
ok. Windows has a thing where it likes each thread to have its own file descriptor open, I don't think I did that in arktika like I did in poolminer
2018-12-23 04:12:59
Fireduck
might be related, but hard to say
2018-12-23 04:13:36
Fable
I should try and balance the info queues with threads per layer right
2018-12-23 04:13:50
Fable
so if they are out of balance i need to change my threads on a layer?
2018-12-23 04:14:12
Fable
Im still only getting 70K~
2018-12-23 04:14:18
Fable
64chunks on each
2018-12-23 04:14:50
Fireduck
yeah, in your case the queues should be about even
2018-12-23 04:14:59
Fireduck
but they will vary a good bit in normal operation
2018-12-23 04:15:24
Fable
but still only 70k, so what is the solution you think?
2018-12-23 04:15:43
Fireduck
are all the CPU cores maxed out at 70k?
2018-12-23 04:15:49
Fable
what linux distro should I use on the rig to run multiple instances
2018-12-23 04:16:25
Fable
and yes
2018-12-23 04:16:26
Fireduck
I like debian, but the only things that actually matter are the kernel and the java version
2018-12-23 04:16:37
Fable
all cpu cores maxed with 512 threads and 4 drives
2018-12-23 04:16:39
Fireduck
so any distro
2018-12-23 04:16:53
Fable
so painful to use on rigs though
2018-12-23 04:17:27
Fireduck
if you don't mind, keep it windows and I'd like to try some things with you over the next few days
2018-12-23 04:17:28
Fable
unless you have everything automated and can control all rigs at once
2018-12-23 04:18:55
Fable
hopefully it can run all the drives with the full field on one instance to save ram usage
2018-12-23 04:19:08
Fable
but that would interfere with how it already is setup I think so
2018-12-23 04:26:52
Fireduck
That would be a pretty intense change to arktika and I think the main problem is about the file descriptors
2018-12-23 04:27:05
Fireduck
So might try something else first