archive snowblossom slack mining
prev 2018-12-06 next
2018-12-06 00:00:07
Rotonen
or, well, you can make .torrents for that as well, both do the same in the end
2018-12-06 00:00:58
Rotonen
the relevant details for how are scattered above, but i think that one example uri should have you covered in how it works
2018-12-06 00:03:49
mjay
with more &dn= it tells me "invalid torrent file"
2018-12-06 00:03:53
mjay
magnet%3A%3Fxt%3Durn%3Abtih%3Aa83c3c8e5a8f68ffb1004dd0e62d380b6fa46805%26dn%3Dsnowblossom.8%26tr%3Dhttp%3A%2F%2Ftracker.1209k.com%2Fannounce%26ws%3Dhttp%3A%2F%2F136.243.82.213%2F%26ws%3Dhttp%3A%2F%2F148.251.122.19%2F%26ws%3Dhttp%3A%2F%2F95.216.9.106%2F%26ws%3Dhttp%3A%2F%2F148.251.177.148%2F%26ws%3Dhttp%3A%2F%2F148.251.10.146%2F%26ws%3Dhttp%3A%2F%2F148.251.40.111%2F%26ws%3Dhttp%3A%2F%2F136.243.71.164%2F%26ws%3Dhttp%3A%2F%2F136.243.90.45%2F
2018-12-06 00:04:08
Rotonen
dn is display name, you can only have one of those
2018-12-06 00:04:24
Rotonen
also you urlencoded the whole thing again so some bits of it are now doubly urlencoded
2018-12-06 00:04:35
Rotonen
you only urlencode the parametres
2018-12-06 00:05:51
mjay
I see
2018-12-06 00:06:14
Rotonen
bits which need to stay literal byte for byte ``` magnet:? xt=urn:btih:<sha256> &dn=<urlencoded display name> &tr=<urlencoded tracker url> &ws=<urlencoded webseed url> &ws=<urlencoded webseed url> ... ```
2018-12-06 00:06:37
Rotonen
it's like any web url where you add query parametres after the `?`
2018-12-06 00:07:00
Rotonen
just that this uri has no location data as to the fetch path as it really does not know where the data is gonna be from that early in the cycle
2018-12-06 00:07:52
Rotonen
so the extra topic (`xt`) will code a `urn:btih` (bittorrent hash) based on which it can try to go and find someone with metadata for that hash
2018-12-06 00:08:14
Rotonen
`dn` (display name) encodes something for the client to show to the end user before it finds the metadata
2018-12-06 00:08:24
mjay
magnet:?xt=urn:btih:a83c3c8e5a8f68ffb1004dd0e62d380b6fa46805&dn=snowblossom.8&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F
2018-12-06 00:08:29
mjay
here we go
2018-12-06 00:08:40
Rotonen
`tr` (tracker) encodes a tracker url, of these there can be many
2018-12-06 00:08:52
Rotonen
and `ws`, as you figured out, configures webseed locations
2018-12-06 00:09:32
Rotonen
so `magnet:` is the same as `http:`, but there's no `//` or `/` prefixed stuff in there, just empty
2018-12-06 00:09:51
Rotonen
and the `?` tail is the same as in web browsers
2018-12-06 00:10:22
Rotonen
and they're joined by a `&` in case of multiple parametres
2018-12-06 00:10:28
mjay
The above link is already working
2018-12-06 00:13:12
Rotonen
``` snowblossom.7 9760e073e3cf6dbe6f2525040f1550b8d3a73eb6 snowblossom.8 a83c3c8e5a8f68ffb1004dd0e62d380b6fa46805 snowblossom.9 09c54657c4e35cea3e86f6c1854a2ae49aedae4d snowblossom.10 7cc6f03a5dea503a9310a2327910b326a751c6e9 snowblossom.11 7f66b2966d92d368ace6160a94cd0304f4766831 snowblossom.7.chunk b24e69043c8e3fa7e313c56c435387c8b61faf1b snowblossom.8.chunk 50d2d8146693167479de2f390fc9914a09940f30 snowblossom.9.chunk a3647e5b5ac9685eb40f738f858e3e976f734ed1 ```
2018-12-06 00:13:19
Rotonen
there's the content hashes
2018-12-06 00:13:32
Rotonen
to make stuff, for the normal fields, the only things you need to change are the hash and the display name
2018-12-06 00:13:58
Rotonen
the chunked ones are a bit silly as @Fireduck made them have the same directory name within the torrent
2018-12-06 00:14:08
mjay
Field 7: magnet:?xt=urn:btih:9760e073e3cf6dbe6f2525040f1550b8d3a73eb6&dn=snowblossom.7&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F Field 8: magnet:?xt=urn:btih:a83c3c8e5a8f68ffb1004dd0e62d380b6fa46805&dn=snowblossom.8&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F Field 9: magnet:?xt=urn:btih:09c54657c4e35cea3e86f6c1854a2ae49aedae4d&dn=snowblossom.9&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F
2018-12-06 00:14:12
Rotonen
but i suppose they'd work as well so long as the main field file name is not the same
2018-12-06 00:14:55
Rotonen
@fydel so maybe you'd also like to offer the web seed magnet uris from above (and add your server[s] to them) instead of the zips of the fields?
2018-12-06 00:15:07
mjay
522MB/s download .. whooo. SSD saturation.
2018-12-06 00:15:32
Rotonen
i'm like 99% certain this is how blizzard seeds out it's game updates
2018-12-06 00:16:21
Rotonen
and they are probably using something like s3 buckets (or cloudfront, or akamai) for the web seeds, but hedging the bet (or cutting the peak off the CDN bill) with letting users volunteer to also upload into the swarm
2018-12-06 00:16:57
Rotonen
and i guess regional ISPs are also pleased with more of the traffic bouncing within their AS instead of saturating their perimeter
2018-12-06 00:17:26
Fireduck
The cheapest I could find was $0.01 per gb transferred
2018-12-06 00:17:49
mjay
aws is crazy expensive when it comes to traffic
2018-12-06 00:18:23
mjay
10$/TB is a lot
2018-12-06 00:19:27
Rotonen
yeah, hetzner was already a pretty good deal at few tens of TB per month for free and 1 euro per TB exceeded
2018-12-06 00:19:33
Rotonen
and now traffic is free
2018-12-06 00:20:20
Fireduck
yeah, I did a dumb on the naming for the chunked ones
2018-12-06 00:20:21
Rotonen
i committed to pay for exceeding the traffic limits already in 2008 and i've never actually gone over the limits
2018-12-06 00:20:26
Rotonen
and i've ran 4 companies on them
2018-12-06 00:20:38
mjay
they never actually charged me for extra traffic. I exceeded the limit regularly and asked to remove it
2018-12-06 00:20:41
Rotonen
@Fireduck wanna redo the chunked ones?
2018-12-06 00:20:48
Fireduck
my plan is to make poolminer read the chunk files and then everyone should use the chunk files for everything
2018-12-06 00:21:16
Rotonen
that would please me at least
2018-12-06 00:21:24
Rotonen
but please make the miners not picky about exact paths
2018-12-06 00:21:29
mjay
why not add a script to split the file into chunks?
2018-12-06 00:21:40
Rotonen
'recursively search within `./snow`' would help people
2018-12-06 00:21:54
Fireduck
yeah
2018-12-06 00:22:14
Rotonen
now people try to config it to point at the big file of the snowfield, the small files of the snow field, the folder of the snow fields, but never the default, which would just work :smile:
2018-12-06 00:34:55
mjay
@Fireduck can you add the magnet links to the snowfields-download-page?
2018-12-06 00:35:34
Clueless
@mjay can you add my torrent server to the magnet url?
2018-12-06 00:35:53
mjay
which one is it?
2018-12-06 00:36:03
Fireduck
Yes
2018-12-06 00:36:36
Clueless
@mjay hm. I need to setup a direct link, but it *will* be `http://torrents.cluelessperson.com`
2018-12-06 00:37:56
mjay
any path, like /announce?
2018-12-06 00:40:22
Clueless
@mjay sorry, I wasn't keeping up with all the convo. exactly what is needed? just a web directory with the files in it?
2018-12-06 00:41:14
mjay
I was about to add your server as a tracker, but for downloads, yes
2018-12-06 00:41:26
mjay
directories with the files
2018-12-06 00:41:44
mjay
find . ./teapot.zip ./snowblossom.9 ./snowblossom.9/snowblossom.9.snow ./snowblossom.9/snowblossom.9.deck.c ./snowblossom.9/snowblossom.9.deck.a ./snowblossom.9/snowblossom.9.deck.b ./snowblossom.8 ./snowblossom.8/snowblossom.8.deck.c ./snowblossom.8/snowblossom.8.deck.b ./snowblossom.8/snowblossom.8.deck.a ./snowblossom.8/snowblossom.8.snow ./index.html ./snowblossom.7 ./snowblossom.7/snowblossom.7.deck.c ./snowblossom.7/snowblossom.7.snow ./snowblossom.7/snowblossom.7.deck.a ./snowblossom.7/snowblossom.7.deck.b
2018-12-06 00:43:07
Rotonen
that's a 404, maybe not add it before there is content
2018-12-06 00:43:18
Rotonen
but you can just add it yourself, all you need is above
2018-12-06 00:43:56
mjay
Field 7: magnet:?xt=urn:btih:9760e073e3cf6dbe6f2525040f1550b8d3a73eb6&dn=snowblossom.7&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F&ws=http%3A%2F%2Fcluelessperson.com%2Ftorrents%2Fsnowblossom%2F Field 8: magnet:?xt=urn:btih:a83c3c8e5a8f68ffb1004dd0e62d380b6fa46805&dn=snowblossom.8&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F&ws=http%3A%2F%2Fcluelessperson.com%2Ftorrents%2Fsnowblossom%2F Field 9: magnet:?xt=urn:btih:09c54657c4e35cea3e86f6c1854a2ae49aedae4d&dn=snowblossom.9&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F&ws=http%3A%2F%2Fcluelessperson.com%2Ftorrents%2Fsnowblossom%2F
2018-12-06 00:44:04
Clueless
@Rotonen I just have to setup the nginx location/directory
2018-12-06 00:44:13
Clueless
doing
2018-12-06 00:44:33
mjay
it is okay if its not ready yet, the client seem to just skip it
2018-12-06 00:45:15
Clueless
@mjay I might've changed it after you saw it, can you double check the directory? :P
2018-12-06 00:45:44
mjay
meh. :stuck_out_tongue:
2018-12-06 00:46:43
mjay
Field 7: magnet:?xt=urn:btih:9760e073e3cf6dbe6f2525040f1550b8d3a73eb6&dn=snowblossom.7&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F&ws=http%3A%2F%2Fcluelessperson.com%2Fshare%2Fpublic%2Ftorrents%2Fsnowblossom%2F Field 8: magnet:?xt=urn:btih:a83c3c8e5a8f68ffb1004dd0e62d380b6fa46805&dn=snowblossom.8&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F&ws=http%3A%2F%2Fcluelessperson.com%2Fshare%2Fpublic%2Ftorrents%2Fsnowblossom%2F Field 9: magnet:?xt=urn:btih:09c54657c4e35cea3e86f6c1854a2ae49aedae4d&dn=snowblossom.9&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F&ws=http%3A%2F%2Fcluelessperson.com%2Fshare%2Fpublic%2Ftorrents%2Fsnowblossom%2F
2018-12-06 00:47:11
Fireduck
We should make a dns entry that resolves to those ips
2018-12-06 00:47:43
mjay
or we just leave them as ips
2018-12-06 00:48:28
Fireduck
Shorter with one hostname for them all
2018-12-06 00:48:43
Fireduck
But probably not super important
2018-12-06 00:49:49
mjay
Few people are going to see this
2018-12-06 00:49:59
Fireduck
True
2018-12-06 00:54:36
Rotonen
rather just list them all by their ip, then you do not have to rely on dns working out for the end user
2018-12-06 00:54:56
Rotonen
also rather have many as then the clients can multi source across many web sources instead of assuming there is only one
2018-12-06 00:55:08
mjay
these IPs are unlikely to change
2018-12-06 00:56:18
Fireduck
ha, the dns is going to work. route 53 is pretty robust.
2018-12-06 00:56:33
Fireduck
cool
2018-12-06 00:57:24
Rotonen
the biggest issue is that most browser / operating system combos do not like to pass long urls from the browser to the os
2018-12-06 00:57:55
Rotonen
on windows and chrome you get about 2k characters worth of space to work with, with ie11, you get about 500 characters
2018-12-06 00:58:13
Clueless
@Fireduck the actual linke doesn't need to be visible at all
2018-12-06 00:58:21
Rotonen
and that's what browsers do if they hit a `scheme:` they do not know how to handle, just pass it to the os
2018-12-06 00:58:44
Clueless
I just do `<a href="actual link">magnet: snowblossom.0</a>` or whatever looks best in the context
2018-12-06 00:58:49
mjay
currently its 472 characters, with encoding
2018-12-06 00:58:55
Rotonen
which can be confusing, but i guess it'll be fine as it just cuts off and good clients will parse it as far as it gets and the first 2k characters will contain the important bits
2018-12-06 00:59:07
mjay
@Clueless directory structure looks okay
2018-12-06 00:59:18
Rotonen
(or it might be it silently errors, behaviours vary)
2018-12-06 00:59:43
Rotonen
@Clueless you could drop field 6 and below
2018-12-06 00:59:48
Clueless
@mjay well, that's directly how it's saved in my archives. `snowblossom -> /mnt/archive_impersonal/torrents/long-term-seeding/snowblossom`
2018-12-06 01:00:22
Clueless
@Rotonen lower fields should probably be kept up for people to test with I think, but I suppose they can just generate them as well
2018-12-06 01:00:27
Clueless
needs to continue working on wikis.
2018-12-06 01:26:32
Rotonen
as if this was not confusing enough already https://www.anandtech.com/print/13677/innodisk-launches-m2-graphics-card-with-4k-output Innodisk Launches M.2 Graphics Card with 4K Output
2018-12-06 02:56:56
Fireduck
Wtf
2018-12-06 03:54:16
Clueless
I mean it is PCIe basically. Right?
2018-12-06 04:05:53
Fireduck
I guess
2018-12-06 08:36:59
fydel
@Rotonen yes. good idea. the zips are silly.
2018-12-06 08:38:00
fydel
and i already have been throttled to 100mbit on that server.
2018-12-06 12:27:11
mjay
do you still want your server on the list?
2018-12-06 12:27:15
mjay
@fydel
2018-12-06 16:37:22
fydel
thanks. i already cancelled that server. but i have another one.
2018-12-06 16:37:44
fydel
@mjay you can add my nürnberg server: http://nur.hamster.science/snowblossom.7/
2018-12-06 16:40:13
mjay
only field 7 @fydel?
2018-12-06 16:41:20
fydel
ja. no more disk space. :disappointed:
2018-12-06 16:41:48
mjay
check your logs if downloads occur
2018-12-06 16:42:40
fydel
yes. "GET /snowblossom.7/snowblossom.7.snow HTTP/1.1" 206 16056320 "-" "Transmission/2.92"
2018-12-06 16:42:50
mjay
Field 7: magnet:?xt=urn:btih:9760e073e3cf6dbe6f2525040f1550b8d3a73eb6&dn=snowblossom.7&tr=http%3A%2F%2Ftracker.1209k.com%2Fannounce&ws=http%3A%2F%2F136.243.82.213%2F&ws=http%3A%2F%2F136.243.90.45%2F&ws=http%3A%2F%2F136.243.71.164%2F&ws=http%3A%2F%2F148.251.40.111%2F&ws=http%3A%2F%2F148.251.10.146%2F&ws=http%3A%2F%2F148.251.177.148%2F&ws=http%3A%2F%2F95.216.9.106%2F&ws=http%3A%2F%2F148.251.122.19%2F&ws=http%3A%2F%2Fcluelessperson.com%2Fshare%2Fpublic%2Ftorrents%2Fsnowblossom%2F&ws=http%3A%2F%2Fnur.hamster.science%2F
2018-12-06 16:42:52
mjay
new link
2018-12-06 16:43:39
fydel
top!
2018-12-06 20:35:28
mjay
Now we have 10gbit for field 7 and 9gbit for field 8 and 9. I think thats plenty
2018-12-06 20:39:40
mjay
Not enough disk space for field 10 .. I have when I delete field 7
2018-12-06 20:39:49
mjay
I'll add it when we are at field 8
2018-12-06 22:28:29
Clueless
who has snowblossom.11 that I can use as a peer?
2018-12-06 22:28:42
Clueless
I only have 4 peers so far.
2018-12-06 22:29:58
Fireduck
I do
2018-12-06 22:30:07
Fireduck
and you probably are pulling it down already
2018-12-06 22:30:17
Fireduck
no one wants that 2tb of crap on their drive
2018-12-06 22:31:04
Fireduck
Peer US 172.92.140.254:30619 Deluge 1.3.15 825.0 KiB/s 0.0 KiB/s
2018-12-06 22:31:06
Fireduck
Is that you?
2018-12-06 23:19:50
Clueless
yup, that's me
2018-12-06 23:20:26
mjay
Snowblossom unique selling point: *Will* fill your drive with an arbitrary amount of crap
2018-12-06 23:20:40
Clueless
eh, I just upgraded to 32TB, so I'm good.
2018-12-06 23:20:46
Rotonen
@mjay *will
2018-12-06 23:23:45
Clueless
sweet, I've got templating down with zabbix