archive snowblossom slack mining
prev 2018-06-28 next
2018-06-28 04:16:06
Tyler Boone
@Shoots, what is your version of java?
2018-06-28 04:20:45
Shoots
I've tried 8, 10 and openjdk 8
2018-06-28 04:20:54
Shoots
I'm using openjdk 8 right now
2018-06-28 04:23:01
Tyler Boone
you had the slower hashing with 10 as well?
2018-06-28 04:24:48
Tyler Boone
and to confirm this is with memfield=true and without setting anything for the precaching
2018-06-28 04:49:01
Shoots
Correct all versions of Java were the same. Tested different versions of poolminer and 1.0.7+ uses less memory but also hashes slower.
2018-06-28 05:10:58
Fireduck
Made some notes for a new mining setup I am thinking about: https://github.com/snowblossomcoin/snowblossom/wiki/Future-Plans
2018-06-28 05:11:07
Fireduck
Extending the idea of hybrid mining to more flexible ends
2018-06-28 07:41:48
Tyler Boone
well, I tried to repro the slowness, but I get the same rate I always got with memory mining. if you take some stacktraces (jstack <pid>) and post them I'll take a look and see if anything looks fishy
2018-06-28 13:29:09
Shoots
getting them right now
2018-06-28 13:29:19
Shoots
dammit this guy is hammering the network with hash now snow:dwmve86sywjhk3xsfznj2wkjjv9j7rnc6z7afgwv
2018-06-28 13:38:56
Shoots
@Tyler Boone this is a different VM its only 32c, but it has the same experience. V1.0.6 PoolMiner gives me 1.75mh V1.1.2 PoolMiner gives me 1.48mh
1.0.6 jstack.txt
2018-06-28 13:40:13
Shoots
1.0.6 uses ~210gb of memory 1.1.2 uses ~155gb of memory
2018-06-28 14:01:41
Tyler Boone
"MinerThread" #107 daemon prio=5 os_prio=0 tid=0x00007f50c4cea000 nid=0x1ae3 runnable [0x00007f13484c3000] java.lang.Thread.State: RUNNABLE at snowblossom.miner.PoolMiner$MinerThread.runPass(PoolMiner.java:324) at snowblossom.miner.PoolMiner$MinerThread.run(PoolMiner.java:394)
2018-06-28 14:01:59
Tyler Boone
I see a bunch of those, and I have no idea why that would be common...
2018-06-28 14:41:20
Shoots
in which version?
2018-06-28 14:47:48
Shoots
oh its in both
2018-06-28 16:42:39
stoner19
damn, one of my itty bitty miners found a block for the pool. Shoulda solo mined :stuck_out_tongue_closed_eyes:
2018-06-28 16:42:59
Fireduck
I thought people would like that block counter
2018-06-28 16:48:40
Tyler Boone
@Fireduck, did you look at that stacktrace from Shoots?
2018-06-28 16:49:02
Tyler Boone
I looked at the code and it's nonsensical in my eyes for that stacktrace to appear so many times
2018-06-28 16:49:20
Tyler Boone
but you've been doing java a lot more than me in the last 5 years. maybe there is some terrible gotcha?
2018-06-28 16:52:01
Fireduck
on merkle_proof.readWord?
2018-06-28 16:52:15
Fireduck
I'd expect things to be inside readword
2018-06-28 16:53:20
Tyler Boone
I know right?
2018-06-28 16:53:32
Tyler Boone
I mean, readWord isn't synchronized
2018-06-28 16:54:18
Fireduck
I think top and java both don
2018-06-28 16:54:28
Fireduck
't have a good way to represent waiting for a page from ram
2018-06-28 16:55:25
Fireduck
ok, on the next gen mining plan, I would love it if we could avoid doing the memory mining in the java process
2018-06-28 16:55:52
Fireduck
linux has really nice shared memory fs and zram or whatever @Rotonen is always on about
2018-06-28 16:55:58
Fireduck
Windows is the trouble there
2018-06-28 16:57:04
Shoots
all this next version mining talk is making me moist
2018-06-28 16:57:35
Shoots
@stoner19 how were you able to see you found the block?
2018-06-28 16:58:39
Fireduck
INFO: Shares: 31 (rejected 0) (blocks 0)
2018-06-28 16:58:43
Fireduck
counter from poolminer
2018-06-28 16:59:41
Shoots
oh Im still using old version
2018-06-28 16:59:52
Shoots
cause you know
2018-06-28 16:59:54
Shoots
hr issues
2018-06-28 17:00:12
Tyler Boone
Windows can do RAM disks. the problem with relying on that kind of stuff is you have to have separate stuff for each platform
2018-06-28 17:00:28
Tyler Boone
or rely on users doing it themselves (hint: disaster)
2018-06-28 17:00:38
Shoots
limit windows users to storage mining only
2018-06-28 17:00:41
Shoots
as punishment
2018-06-28 17:00:57
stoner19
``` [2018-06-28 14:38:54] INFO snowblossom.miner.PoolMiner printStats 15 Second mining rate: 588419.267/sec - at this rate 3.802 minutes per share (diff 27.000) [2018-06-28 14:38:54] INFO snowblossom.miner.PoolMiner printStats 1-min: 588.168K/s, 5-min: 587.713K/s, hour: 588.002K/s [2018-06-28 14:38:54] INFO snowblossom.miner.PoolMiner printStats Shares: 288 (rejected 0) (blocks 1) ```
2018-06-28 17:01:02
Tyler Boone
that is counter to the purpose of snowblossom
2018-06-28 17:01:06
Fireduck
or they can figure out their own ramfs
2018-06-28 17:01:31
Fireduck
but Tyler is right
2018-06-28 17:01:36
Tyler Boone
sounds like a killer way to attract more users
2018-06-28 17:01:56
stoner19
payout from the pool for the block I solved was 0.96. LOL that blows
2018-06-28 17:02:09
Tyler Boone
pwned
2018-06-28 17:02:13
stoner19
no joke
2018-06-28 17:02:18
Fireduck
workin for the man
2018-06-28 17:02:36
Fireduck
Don't want to mine on Maggie's Mining Pool no more
2018-06-28 17:03:21
stoner19
:heart:
2018-06-28 17:04:14
Tyler Boone
https://www.youtube.com/watch?v=b2F-DItXtZs YouTube Video: Episode 1 - Mongo DB Is Web Scale
2018-06-28 17:04:21
Tyler Boone
that will make you want to work on a farm
2018-06-28 17:04:43
Fireduck
I die
2018-06-28 17:05:36
Tyler Boone
why the hell does java force us to specify a max heap size that is always and forever the max for that process????
2018-06-28 17:05:46
Tyler Boone
what kind of moron designs this shit?
2018-06-28 17:06:08
Fireduck
hard to say
2018-06-28 17:07:07
Tyler Boone
The biggest problem with Java is that the designers think they are smarter than the users, and they force their way on people
2018-06-28 17:07:15
Tyler Boone
Larry Wall FTW!
2018-06-28 17:09:15
Fireduck
Weren't smart enough to stop the OOP shitstorm
2018-06-28 17:09:23
Fireduck
but people can write terrible in any language
2018-06-28 17:10:19
Tyler Boone
define: "OOP shitstorm"
2018-06-28 17:10:29
Tyler Boone
@Slackbot, what do you do?
2018-06-28 17:10:42
Tyler Boone
@Slackbot, tell me a joke
2018-06-28 17:10:50
Tyler Boone
worthless
2018-06-28 17:11:03
Tyler Boone
@Slackbot help
2018-06-28 17:11:05
Fireduck
OOP shitstorm, where people don't type any code without making an interface and a factory
2018-06-28 17:11:16
Fireduck
slackbot tells me when people sign up using my subscribe link
2018-06-28 17:11:18
Tyler Boone
Java encourages such behavior
2018-06-28 17:11:38
Fireduck
I think java is a fine language, it is the programmers who use it that are wrong
2018-06-28 17:11:51
Tyler Boone
I'm not sure how you would instantiate that factory though without a FactoryInstantiator
2018-06-28 17:12:14
Fireduck
and a carry all to move it if a sandworm comes
2018-06-28 17:13:29
Tyler Boone
nothing has ever been so beautify
2018-06-28 17:14:27
Fireduck
so the largest field imaginable is lets say 16TB, that would be 16 thousand of these files
2018-06-28 17:14:36
Fireduck
pushing the open file limit, but whatever at that point
2018-06-28 17:14:43
stoner19
wow `dwmve86sywjhk3xsfznj2wkjjv9j7rnc6z7afgwv` is murdering these blocks
2018-06-28 17:15:35
stoner19
everyone has to run out of money for AWS and google compute eventually, right?
2018-06-28 17:15:57
Fireduck
I have
2018-06-28 17:16:22
stoner19
seems other haven't yet. Unless they're planning on just getting their account terminated after maxing it out and not paying their bill.
2018-06-28 17:16:40
Fireduck
haha
2018-06-28 17:17:22
Tyler Boone
that's how I roll with my mortgage, so it's probably cool
2018-06-28 17:17:55
Fireduck
yeah, it will take them a while to find you
2018-06-28 17:18:04
Fireduck
2nd ave NW or whatever, that could be anywhere
2018-06-28 17:18:44
Fireduck
so I am thinking rather than decimal chunk numbers I'll do 4 hex digits
2018-06-28 17:18:51
Fireduck
.0000 to .FFFF
2018-06-28 17:19:17
Tyler Boone
nothing matters
2018-06-28 17:19:37
stoner19
take out a second mortgage to pay for 48 hours worth of AWS instances
2018-06-28 17:19:37
stoner19
nice
2018-06-28 17:21:00
Fireduck
what is an easy way to convert a number into hex in bash?
2018-06-28 17:22:06
Rotonen
@Fireduck also ext3 can only do 64k subitems per dir
2018-06-28 17:22:33
Rotonen
@Fireduck as in filecount issues are something of yesteryears, no sense to worry
2018-06-28 17:23:20
Rotonen
@Fireduck use a python oneliner for that in your script
2018-06-28 17:23:27
stoner19
`printf '%x\n' <decimal>` ?
2018-06-28 17:23:59
Fireduck
I dig that
2018-06-28 17:24:11
Fireduck
I don't actually know python worth shit
2018-06-28 17:24:24
Fireduck
and using bash to just run dd commands to split out chunks
2018-06-28 17:24:25
Rotonen
prone to such input maliciousness, but sure
2018-06-28 17:26:31
Tyler Boone
we should just rewrite everything in Perl
2018-06-28 17:27:55
Fireduck
perl yeah
2018-06-28 17:28:31
Rotonen
surprisingly nice stuff in the ecosystem
2018-06-28 17:29:06
Fireduck
perl? It has been around long enough, so one would hope
2018-06-28 17:30:13
Tyler Boone
Perl attracted all the best hackers in the 90s and 2000s, and had one of the first really good community library repositories in CPAN
2018-06-28 18:37:41
Shoots
@stoner19 we should start getting more blocks, just doubled up the pool hr
2018-06-28 18:45:47
Rotonen
i’m alone on mine and i still got a block
2018-06-28 18:46:18
Shoots
as long as you're hashing it all averages out over time
2018-06-28 18:57:59
AlexCrow
Which pool ? @stoner19 we should start getting more blocks, just doubled up the pool hr
2018-06-28 19:01:45
Shoots
proto
2018-06-28 19:03:47
AlexCrow
and what is avr. hr now on protopool ?
2018-06-28 19:31:08
Shoots
I think its around 120mh only right now
2018-06-28 19:33:11
AlexCrow
ok, I'll put ~ 250MH on protopool
2018-06-28 19:33:57
Fireduck
Fatcats
2018-06-28 19:34:08
Fireduck
I think I have about 150kh
2018-06-28 19:34:43
AlexCrow
I think you was one of the first miners - so it's enough)
2018-06-28 19:35:16
Fireduck
Should hope I was one of the first
2018-06-28 19:41:45
Shoots
Alex you're the one who keeps jumping on our pool every now and then I assume?
2018-06-28 19:48:42
AlexCrow
I did it few times)
2018-06-28 19:53:56
Shoots
we appreciate it, didnt take long to find some blocks now
2018-06-28 20:05:37
bl0ckchain
250mh that’s about 50-100 server whoa :open_mouth:
2018-06-28 20:06:00
Fireduck
Or they broke the pow
2018-06-28 20:06:11
Fireduck
But I think it is servers
2018-06-28 20:10:24
Tyler Boone
if you broke bitcoin's PoW, how much money do you think you could extract before people figured it out and bitcoin's value crashes to zero?
2018-06-28 20:10:54
Fireduck
I fuckton if not dumb
2018-06-28 20:11:22
Fireduck
Mine at like 20% total network rate and keep mouth shut
2018-06-28 20:12:19
Tyler Boone
problem is extracting actual money without someone looking into it
2018-06-28 20:12:39
Fireduck
Ah
2018-06-28 20:12:52
Tyler Boone
a single client pulling out hundreds of millions from an exchange is going to attract a lot of attention
2018-06-28 20:13:17
Tyler Boone
pulling in more people increases risk of secret leaking
2018-06-28 20:13:34
Fireduck
Yeah, don't mine more than you can trickle out
2018-06-28 20:13:43
Fireduck
Probably a million a week
2018-06-28 20:13:49
Shoots
@bl0ckchain I imagine its more like 150 servers
2018-06-28 20:14:10
Shoots
depending on the instance type
2018-06-28 20:14:18
Shoots
I have 30 running and its 55mh
2018-06-28 20:14:32
Tyler Boone
Someone before said they were getting 5Mh/s per server, so 50 of those would be 250
2018-06-28 20:14:41
stoner19
I have 2 at 1.1M/s total
2018-06-28 20:14:47
Tyler Boone
obviously some servers are more equal than other servers
2018-06-28 20:19:50
AlexCrow
it's 160x32cores. I get only ~ 1.8mh per server Someone before said they were getting 5Mh/s per server, so 50 of those would be 250
2018-06-28 20:21:22
Tyler Boone
what is 160?
2018-06-28 20:21:53
Fireduck
160gb
2018-06-28 20:21:56
Tyler Boone
my 6 core machine gets 660kh when memory mining. I'm guessing your server is hitting memory bus limits
2018-06-28 20:22:21
Tyler Boone
or you forgot to increase the thread count in the config
2018-06-28 20:22:57
AlexCrow
I have 64 threads in config, and use 180gb ram
2018-06-28 20:24:26
AlexCrow
160 servers, 32cores and 180gb per server what is 160?
2018-06-28 20:28:44
Shoots
whats your hr per vm? and which version of miner are you running?
2018-06-28 20:28:58
Shoots
nvm I see now
2018-06-28 20:29:31
Shoots
I cant get 1.8mh with only 180gb mem usage, so wondering how you get that, I have an issue with the old version using too much memory and the new version using less but not providing the same hr
2018-06-28 20:29:52
Shoots
in order to get 1.8mh I have to run 1.0.6 and it uses 210gb of mem
2018-06-28 20:30:10
AlexCrow
I run 1.1.2
2018-06-28 20:30:29
Shoots
weird, I wonder why I get such low hr
2018-06-28 20:30:33
Shoots
which version of java?
2018-06-28 20:30:42
Shoots
aws or gcp?
2018-06-28 20:31:10
AlexCrow
I tried java 8 and 10, same HR GCP
2018-06-28 20:31:27
Shoots
hmm Im on aws
2018-06-28 20:31:45
Shoots
it gets up to full speed and then just keeps creeping the memory up and up and up on 1.0.6
2018-06-28 20:32:29
Shoots
on 1.1.2 it uses about 150gb of mem, but it doesnt fully utilize all the cores, seems to be bottlenecked by something as if the threads Ive set in the config arent actually all fully running
2018-06-28 20:32:40
AlexCrow
I run miner with these params " --jvm_flags=" -Xms"$xms"g -Xmx"$xmx"g -XX:+OptimizeFill -XX:+AggressiveOpts -XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent -XX:+ParallelRefProcEnabled -XX:+UseStringDeduplication -XX:+UnlockExperimentalVMOptions -XX:G1NewSizePercent=20 -XX:+UnlockDiagnosticVMOptions -XX:G1SummarizeRSetStatsPeriod=1" configs/miner-mainnet.conf "
2018-06-28 20:32:54
AlexCrow
xms =174 xmx 174
2018-06-28 20:33:03
Shoots
oh wow ok let me try
2018-06-28 20:33:12
stoner19
oh...
2018-06-28 20:33:20
stoner19
flags you say...
2018-06-28 20:33:36
AlexCrow
I don't understand nothing in Java
2018-06-28 20:33:56
AlexCrow
so all these params it's just ....
2018-06-28 20:33:59
Shoots
when it doubt flag it out
2018-06-28 20:34:22
AlexCrow
which one ?
2018-06-28 20:35:16
stoner19
I have the zip. I'd have to compile from the repo to use those flags I think
2018-06-28 20:35:25
stoner19
actually probably not. Just put the flags in the bash script, eh?
2018-06-28 20:36:09
AlexCrow
it's running flags /PoolMiner --jvm_flags= " " config
2018-06-28 20:37:34
Shoots
compiling from source gives me a bit better hashrate than the pre compiled, but still not as much as 1.0.6 for some reason
2018-06-28 20:38:00
AlexCrow
yeah binary got me only 1.4MH
2018-06-28 20:38:01
stoner19
I actually got quite a bit less from compiling
2018-06-28 20:44:05
stoner19
yeah, it doesn't like the flags `SEVERE: Incorrect syntax. Syntax: PoolMiner <config_file>`
2018-06-28 20:44:49
Shoots
you cant use the same syntax as the pre built script
2018-06-28 20:44:57
Shoots
it uses java -jar
2018-06-28 20:45:39
Fireduck
You guys are all nuts, I love it
2018-06-28 20:45:59
Shoots
this is what Im using
2018-06-28 20:46:12
stoner19
bash script is nothing more than `java -jar PoolMiner_deploy.jar`
2018-06-28 20:46:28
Shoots
you can either run with java -jar filename
2018-06-28 20:46:44
Shoots
or ./PoolMiner --jvmflags=
2018-06-28 20:47:08
stoner19
will give it a try again. I was trying to put the flags into the bash script
2018-06-28 20:47:20
Shoots
you can put them in the bash script
2018-06-28 20:47:22
Shoots
look at mine above
2018-06-28 20:48:05
stoner19
strange, wonder why mine is being crabby. Will keep trying.
2018-06-28 20:48:17
Shoots
actually I didnt copy and paste the whole thing, its missing the end
2018-06-28 20:50:59
Shoots
still only getting 1.45mh with 1.1.2 and all those flags
2018-06-28 20:53:25
Fireduck
Have you guys tried using shm filesystem rather than memfield?
2018-06-28 20:53:33
Shoots
nope
2018-06-28 20:54:47
Fireduck
tmpfs /var/shm tmpfs defaults,noexec,nosuid,size=160G 0 0
2018-06-28 20:54:52
Fireduck
In fstab
2018-06-28 20:55:13
Shoots
oh wait yes I did
2018-06-28 20:55:23
Shoots
and it was taking god awfully long to load into mem so I just gave up
2018-06-28 20:55:31
Fireduck
Ha
2018-06-28 20:55:47
Shoots
I restart my miners too often to have them down for an hour copying the field into mem
2018-06-28 20:55:47
Fireduck
If you run on gcp I have a bucket with the snowfield
2018-06-28 20:55:58
Shoots
I dont know why it was taking so much longer than using memfield option
2018-06-28 20:56:16
Fireduck
Memfield lazy loads as needed
2018-06-28 20:56:29
Fireduck
So probably takes that hour to get to full speed
2018-06-28 21:17:23
stoner19
tiny tiny tiny bit of improvement with those flags. Thanks for sharing @AlexCrow & @Shoots
2018-06-28 21:34:48
fydel
yes. thanks. i have a ~10% better hashrate now.
2018-06-28 21:36:44
AlexCrow
1.1MH vs 1.8MH with memfield tmpfs /var/shm tmpfs defaults,noexec,nosuid,size=160G 0 0
2018-06-28 21:37:00
Rotonen
@AlexCrow it’s not the cores, it’s the unthrottled cores per memory channels ratio
2018-06-28 21:38:39
Rotonen
and striping shm guaranteedly across channels on the cloud is quite the dark art
2018-06-28 21:38:48
Rotonen
YMMV
2018-06-28 21:40:15
Rotonen
1) linux does not yet have NUMA aware shared memory 2) cloud vendors do not expose translucent NUMA backendy stuff
2018-06-28 22:01:18
Fireduck
Anyone who wants to bleed on the edge of new tech, download this: https://snowblossom.org/snowfields/snowblossom.7.chunk.torrent
2018-06-28 22:01:24
Fireduck
splits the field into 1gb chunks
2018-06-28 22:01:34
Fireduck
hopefully the code to read that will be read in the next few days
2018-06-28 22:07:26
Shoots
sweet
2018-06-28 22:08:06
Shoots
so I used those flags and it didnt improve my hr on 1.1.2, butttt it reduced the memory usage on 1.0.6 so i can now run my 48 core vm that has 185gb of memory, should get better hash per $ now
2018-06-28 23:22:23
Rotonen
@Fireduck neat