rieMiner - Solo + pooled Riecoin mining

[Rockhawk]

Rockhawk, how did you find the 128 MiB + 650 MiB*SieveWorkers terms for the memory estimation?
It seems to work good enough for big Sieves, but the estimation is imprecise for very low Prime Table Limits as it estimates ~800 MiB, while I am able to run rieMiner (from the Light Branch) without swapping in a Pentium III computer with only 384 MiB of RAM...
Any way to get a more precise estimation in these extreme cases? This might still be useful for people willing to play with rieMiner in a RaspBerry Pi for example (such thing will still find very easily blocks on TestNet, a Pentium III is needing less than or about 10 minutes)...
And what was the 6-tuple record 20 years ago :D ? I have some vintage computers, it is always interesting to see what they can do with the software of today.

The estimation was just a rough guess based on a little testing. The main source of error though is the offset tables, so you could just compute the estimate after computing the actual size of them, given we are no longer restricting the PTL based on memory usage.

It looks like 20 years ago the record was 172 digits, I got a 213 digit record in 2000 and a 290 digit one in 2001 (that one using a couple of friend's machines in addition to my own).

Also, what do you think about the riePool's claim of a miner 20-25% faster than fastrie (closed source)?
Legit or simple false advertising? If this were true for equal settings, it would be even better than rieMiner now.
He even pretended to have written a GPU miner 4.5x faster. As an expert in this domain, do you think this possible?
The facts are still that this is the biggest Riecoin pool for now, not sure if it is because people are thinking that they are mining faster and earning more with this software.

20-25% faster is not impossible - we're over that if you account for the increased speed due to higher sieve limits and I'm sure some increases in raw speed are still possible (although I'm out of ideas right now).

I don't know what 4.5 times faster using a GPU miner means - if he simply means comparing 100% CPU, no GPU to 100% CPU plus 100% GPU with a beefy graphics card, that may be possible.

Did you ever get to the bottom of those crashes? My miner has been running in debug for 324 hours now with no issues.

Just to let you know, I'm likely to remain mostly out of contact until the second week of January.

[Rockhawk]

There is something weird with 32 bits builds though... The performance is much worse, like 4x slower, even on a same Windows computer (that can run both), not sure why. I do not really care as not much people will mine with 32 bits computers, but still. I was unable to find the issue, even replacing UInt64_ts by UInt32_ts did not help. Or maybe it is normal?

I assume this is just that you are using a 32-bit version of GMP? A 32-bit limb size will significantly increase the number of calculations you need to do for each multiprecision arithmetic operation.

As we've seen, the 32-bit AVX implementations can get a little faster than a 64-bit implementation by operating on 8 candidates simultaneously, but being able to do full 64-bit wide multiplies wins you a lot.

[Pttn]

For some reason, I never experienced crashes again since the AVX code... This is really weird, I wonder what happened. But as you saw, the stable version was released.

Interesting to know that 20 years ago, we would have destroyed in minutes the record with the current software :D.
Who knows if someone will find a new algorithm 10x faster than the current one. I wonder what quantum computers could do here 20 years later as well.

I see, I wonder what we could get from things like Ryzen/i7 + 2-4x RTX 2080 if we were to write GPU code... But the power consumption would also be huge.

Ok, I guessed that 32 bits would slower the operations, but not by that much. Maybe 20-30%, not 75%...

Happy Holidays!

[Pttn]

Up to 10000 RIC bounty for anyone that writes elegant code for a fourth mode (with Solo/Pool/Benchmark) that can be used to beat new records for different tuple lengths and makes rieMiner a reference software for prime constellation finding.

Read this issue.

Other issues were also open if anyone if interested in clearing them.

[Rockhawk]

Up to 10000 RIC bounty for anyone that writes elegant code for a fourth mode (with Solo/Pool/Benchmark) that can be used to beat new records for different tuple lengths and makes rieMiner a reference software for prime constellation finding.

Read this issue.

Other issues were also open if anyone if interested in clearing them.

Cool! As you may be aware I have started work on such a mode in the feature-deepsieve branch in my repo. Currently I don't have much time to work on it, though my planned next step was going to be merging in your recent refactoring and making it a mode.

[Pttn]

I think that the bounty is basically made for you as likely not many people will even know about it, let alone try to get it. My main hope is that once the rieMiner name appears in the world record list, we attract more developers and RIC buyers/holders. And why not, get academic interest, and researchers that find and implement even better algorithms in rieMiner.

For now, we cannot use directly rieMiner to beat 7-tuples and longer records. Mining directly with (0, 2, 6, 8, 12, 18, 20) offsets in the configuration is very inefficient (this needs to be fixed).
Though, it looks like that the 7-tuple record could be beaten fairly easily, it is currently at Difficulty ~1715, and as it would take ~2 days to find a block (6-tuple) at difficulty ~1750, with ratio of about 30, we can expect to beat easily that record in ~2 months with a 2700X and the current implementation, by mining (0, 2, 6, 8, 12, 18) "constellations", using compatible offsets, and checking manually if any block is also a 7-tuple.
This workaround could also be used to beat 8-tuples and longer, though rieMiner should (must, for the bounty) be fixed to work directly by putting the correct constellation type offsets in the configuration file.

Writing a very detailed document about the current algorithm would be useful as well. Honestly I only have a superficial understanding of what the miner is actually doing.

Cool! As you may be aware I have started work on such a mode in the feature-deepsieve branch in my repo. Currently I don't have much time to work on it, though my planned next step was going to be merging in your recent refactoring and making it a mode.

Yes I have seen it, though I did not really test it as it ate my 32 GB of RAM, and I did not look where is the option to limit memory usage. Also as it was based on a old commit, I assumed that for now it is more a draft than anything else and did not look further. I will test again once you implemented it as a fourth mode; I do not plan to commit anything soon so you can start from Commit 172 without issue.

Did you get yourself satisfying results with your deepsieve branch code?

[Rockhawk]

Why do you say mining directly with a (0, 2, 6, 8, 12, 18, 20) configuration is very inefficient? The sieving for non 6-tuples is very slightly worse because it doesn't use the AVX code, but the difference is very small.

I get what I believe to be reasonable results mining for a 7-tuple of difficult 1750 with

Code: Select all

Tuples = 5
PN = 194
POff = 15760091, 25658441
ConsType = 0, 2, 4, 2, 4, 6, 2

plus a change in the miner to record the full N for any tuples found (as submit block won't work as it assumes only the low 256-bits are important). With this config my machine looks like it would find a record in about 3 months. (Those POff are chosen so there's a remote possibility that any 7 tuple might be an 8 tuple).

Note that if you just mine for 6 tuples and then test for a 7th prime then your ratio on getting hits on that last prime will be awful as you haven't sieved it, and doing the extra sieving makes your candidates noticeably sparser which slows down the mining.

Did you get yourself satisfying results with your deepsieve branch code?

It is a bit of an improvement for higher tuple lengths, but it isn't as much as I was hoping - I just did a test run on the proposed 7-tuple of difficulty 1750 and it's only about 10% faster. I'll have to think again on the best approach.

[Pttn]

Why do you say mining directly with a (0, 2, 6, 8, 12, 18, 20) configuration is very inefficient? The sieving for non 6-tuples is very slightly worse because it doesn't use the AVX code, but the difference is very small.

I get what I believe to be reasonable results mining for a 7-tuple of difficult 1750 with

Code: Select all

Tuples = 5
PN = 194
POff = 15760091, 25658441
ConsType = 0, 2, 4, 2, 4, 6, 2

plus a change in the miner to record the full N for any tuples found (as submit block won't work as it assumes only the low 256-bits are important). With this config my machine looks like it would find a record in about 3 months. (Those POff are chosen so there's a remote possibility that any 7 tuple might be an 8 tuple).

Note that if you just mine for 6 tuples and then test for a 7th prime then your ratio on getting hits on that last prime will be awful as you haven't sieved it, and doing the extra sieving makes your candidates noticeably sparser which slows down the mining.

I see, I was always using a PN of 40 and got really awful results with that, as there is an insane CPU Underuse. That is why I said that it was unoptimized (I did not care as it at least worked and Riecoin is focused on 6-tuples). Things seem much better with PN 194 and the speed seem to match the one that I get with truncated tuple.
In this case, is a bigger PN mandatory for longer tuples? I thought that it should work as fine as 6-tuples if I used the same settings (minus absence of 6-tuple specific optimizations).
If so then I retract everything I said regarding unoptimization. Is there an optimal PN for each length then?

Did you get yourself satisfying results with your deepsieve branch code?

It is a bit of an improvement for higher tuple lengths, but it isn't as much as I was hoping - I just did a test run on the proposed 7-tuple of difficulty 1750 and it's only about 10% faster. I'll have to think again on the best approach.

For the bounty, I am hoping for improvements of more than 50% for 6 tuples and longer.
If we can't do more for now it is what it is but in this case it will not be enough for the bounty (or I might still reward less for a 30-50% improvement).

[Rockhawk]

I see, I was always using a PN of 40 and got really awful results with that, as there is an insane CPU Underuse.

Ah, yes I see that too. The problem is simply that there aren't enough candidates left after sieving to keep the prime testing threads busy, which is the same problem I hit when going for longer tuple lengths, and the deepsieve branch does avoid this.

Is there an optimal PN for each length then?

Increasing the primorial number increases the density of candidates, so the same amount of sieving work gets you more candidates to test. The optimal primorial number is always the largest that fits in your difficulty - that's why many record k-tuplets are of the form k.n#+<constellation>.

[Pttn]

I just got some Samsung B Die DDR4...
Unfortunately (or fortunately), the algorithm does not benefit from faster RAM...
Only ~1% increase from 2400 CL15 to 3200 14-14-14-34! Even in a Ryzen platform, where there should be a neat boost!

235.9 pps vs 233.4 pps in Standard Benchmark (runs with 100000 2-tuples).

So if someone wants to optimize a build for Riecoin, don't waste money on faster memory. Just take some Value 2133-2667 MHz and you will be fine.
However, more quantity is significantly better (32 vs 16 GB) to support higher PTLs.