rieMiner - Solo + pooled Riecoin mining

Riecoin mining software & pools
Post Reply
Pttn
Posts: 131
Joined: 24 Aug 2018, 13:37

Re: rieMiner - Solo + pooled Riecoin mining

Post by Pttn » 12 Nov 2018, 21:54

Good job! I tested it, and the CPU Underuse seems to be much less important in the many cases I tested, but still significant (for Sieve 2^32 and current mining dificulty). For Sieve 2^31, it became negligible. Also, when I mine at Difficulty 304 (Testnet difficulty), it is still a major problem, the CPU is only used ~25% with Sieve 2^24 to 2^31, and ~50% with Sieve 2^20. I need to reduce it to 2^16 to max out the 16 threads usage.

For difficulty 304, having one 16 threads instance with Sieve 2^16 (CPU is maxed out) gives me ~51440 1t/s with ratios ~10.6 while using two 8 threads instances with Sieve 2^19 gives me ~48800 1t/s with ratios ~9.0. For difficulty 1400, the impact is much less important, with ~347 1t/s with ratio ~25.0 (1 x Sieve 2^31) versus ~339 1t/s with ratios ~24.4 (2 x Sieve 2^32). In the second case, only the first ratio is significant, while the 3 first ones converged for the former case. It has been tested that all the ratios seems to be constant (1 t/s to 2 t/s and 5 t/s to 6 t/s will be the same if time tends to infinity).
Take the ratios to power 5 to estimate 6-tuples find rate and to see that the mining is actually (way) faster with slightly worse 1 t/s metric and better ratio: more than 2x faster for Difficulty 304, and ~10% faster in the practical case.

Maybe the CPU Underuse Bug will become negligible as Riecoin finds new miners to reach Difficulty > 1600 again, but as I still get better results by running multiple instances, the CPU underuse fix is definitively only partial and needs to be completed. Also, if someone had a (or even bi/quad) 16C/32T CPU or more, this issue will probably still be huge for them (for people reading this, if someone owns a CPU with 32 threads or more, I am interested to see how much CPU Underuse it suffers while using Sieves >= 2^31 for current mining Difficulties).

One good side effect of your latest pull request is that it seems to have improved a bit the performance in both actual mining and Benchmark Mode.

I donated 1000π RIC for the optimizations and this partial fix :D! I would gladly donate some more if you fixed the CPU Underuse completely. In particular, try to make mining at Testnet Difficulty (304) always max out the CPU, even for Sieves >= 2^30 (without any performance loss, of course).

Hopefully the price recovers in near future, you deserve it and Riecoin is severely undervalued. Why are there still stupid people selling?...
Last edited by Pttn on 12 Nov 2018, 22:33, edited 1 time in total.
rieMiner - Riecoin solo + pooled miner
Personal Riecoin page (links, download,...)
freebitco.in - earn up to $200 in BTC each hour!

Pttn
Posts: 131
Joined: 24 Aug 2018, 13:37

Re: rieMiner - Solo + pooled Riecoin mining

Post by Pttn » 12 Nov 2018, 21:58

And now, let's do the official release announcement of rieMiner 0.9β3! It is a big one!

Direct download link of the official Win64 binary.

- Assembly optimizations from Michael Bell (Rockhawk). Performance increase is variable depending on your computer and the rieMiner configuration and hard to measure, but should be at least ~5% and can be much more. Note that this version also uses more memory;
- Partial CPU Underuse Bug fix, also from Rockhawk: this problem is significantly less important, but still present;
- Now possible to use custom configuration file paths;
- Now possible to save tuples counts by difficulty for statistical purposes;
- Some aesthetic changes like showing 1 to 3-tuples/s metrics instead of 2 to 4;
- Includes support of other constellations types to be ready in case of a POW change in the future, or just to be able to find different prime constellations;
- Some other minor changes.
rieMiner - Riecoin solo + pooled miner
Personal Riecoin page (links, download,...)
freebitco.in - earn up to $200 in BTC each hour!

Rockhawk
Posts: 48
Joined: 29 Oct 2018, 21:12

Re: rieMiner - Solo + pooled Riecoin mining

Post by Rockhawk » 12 Nov 2018, 23:02

Awesome to see the release, and thank you for the generous donation!

Understood that you're still seeing a problem with 16 threads at current mining difficulty - improving the balance of work queued to the worker threads is getting pretty tricky at this stage. You might be able to get it close to stable on 16 threads by increasing the proportion of sieve threads (set at the top of the miner init).

However, for a general fix I think it's going to be simplest to just instantiate two or more Miner instances sharing the same parameters and precomputed data, so effectively allow you to run as many instances as you like with much lower overhead. I'll have a go at that, probably tomorrow.

One thing I noticed is that the optimal sieveBits value varies between different machines.

And thanks for adding the configurable config file name - that should make it nice and easy to put together a standard set of verification configs to check performance is remaining stable (or improving!) as we make changes.

Pttn
Posts: 131
Joined: 24 Aug 2018, 13:37

Re: rieMiner - Solo + pooled Riecoin mining

Post by Pttn » 14 Nov 2018, 01:09

I tested in advance your multiple instances code, and sometimes, there will be a concurrent access problem at start:

Code: Select all

Will use 8 instances each with 4 threads.
-----------------------------------------------------------
[0000:00:00] Started mining at block 1, difficulty 304
, difficulty 304
Tuples found for diff 304: (0 0 0 0 0 0) during 0.000 s
Tuples/s: (0.000000 0.000000 0.000000 0.000000 0.000000 0.000000)
Ratios: (inf inf inf inf inf)
[0000:00:04] 6-tuple found
And it seems that there is a performance loss over running manually multiple instances. Maybe this fix is not worth it after all (or the fix itself needs to be fixed). As you still did not any pull request, maybe you are even already working on it.
rieMiner - Riecoin solo + pooled miner
Personal Riecoin page (links, download,...)
freebitco.in - earn up to $200 in BTC each hour!

Rockhawk
Posts: 48
Joined: 29 Oct 2018, 21:12

Re: rieMiner - Solo + pooled Riecoin mining

Post by Rockhawk » 14 Nov 2018, 14:54

Cool :)

Yes I agree that it would be better to be able to specify the the instances separately, I'll make that change.

Thanks for spotting that locking issue. I haven't done any performance comparisons yet, but I don't see any good reason why this version shouldn't be able to have the same perf as multiple instances. Hopefully the difference you're seeing is due to a bug, I'll investigate this evening.

Pttn
Posts: 131
Joined: 24 Aug 2018, 13:37

Re: rieMiner - Solo + pooled Riecoin mining

Post by Pttn » 14 Nov 2018, 20:40

Rockhawk wrote:
14 Nov 2018, 14:54
Yes I agree that it would be better to be able to specify the the instances separately, I'll make that change.
Oops, so you read before I edited... Another solution came to my mind and I planned to do it myself after merging, but while we are at it: not adding a new key Instances, but instead use the Threads one like for the Constellation Type: specify exactly how many threads for each instances.

For example,

Code: Select all

Threads = 6, 6, 4
Will launch two instances of 6 threads and one of 4 threads.
rieMiner - Riecoin solo + pooled miner
Personal Riecoin page (links, download,...)
freebitco.in - earn up to $200 in BTC each hour!

Rockhawk
Posts: 48
Joined: 29 Oct 2018, 21:12

Re: rieMiner - Solo + pooled Riecoin mining

Post by Rockhawk » 14 Nov 2018, 21:10

Pttn wrote:
14 Nov 2018, 01:09
And it seems that there is a performance loss over running manually multiple instances. Maybe this fix is not worth it after all (or the fix itself needs to be fixed). As you still did not any pull request, maybe you are even already working on it.
I've done some testing on various difficulties and I'm seeing pretty much identical results with 2 instances in the config vs 2 copies running side by side. With what configuration are you seeing the discrepancy (what difficulty, sieve, num threads, num instances)? Are you starting and stopping the multiple copies of rieMiner at the same time?

I think this change should be a big win because with the instances running in the same copy of the program you can sieve higher.

I've pushed a fix to the locking problem, and the change to make number of instances configurable.

Rockhawk
Posts: 48
Joined: 29 Oct 2018, 21:12

Re: rieMiner - Solo + pooled Riecoin mining

Post by Rockhawk » 14 Nov 2018, 21:12

Pttn wrote:
14 Nov 2018, 20:40

Code: Select all

Threads = 6, 6, 4
Will launch two instances of 6 threads and one of 4 threads.
OK, I've committed the simple version already, I'll let you do the complex one :)

Pttn
Posts: 131
Joined: 24 Aug 2018, 13:37

Re: rieMiner - Solo + pooled Riecoin mining

Post by Pttn » 14 Nov 2018, 23:59

I just performed this test (stop after 20000 2-tuples, Difficulty 1400):

2 rieMiner 0.9β3.1 instances of 8 threads, Sieve 2^32 : ~3.30 blocks/days (this new metric should be the most precise and appropriate for benchmarking). In details: ~346 primes/s with ratio ~24.6. Sums/averages of the 2 instances. I start them the same time, and they end with a slight difference, but the skew from the delay is negligible (and easily correctable).

Vs 1 rieMiner instance with your code, containing 2 instances of 8 threads, Sieve 2^32: ~3.09 blocks/days. In details: ~331 primes/s with ratio ~24.7.

Both should get exactly the same benchmark result, minus the statistical error. The statistical error for testing this long should be at worst ~5 1-tuples/s metrics, so there is clearly a flaw somewhere.

Yesterday, I tested this multiple times, though quicker, and already got this conclusion. Always a few % loss, and never in the other side (which would happen half of the times if the performance is actually the same). I do not think that it is a coincidence that I get worse results with your code.

With Difficulty 400 and Sieve 2^16: 16334 pps/r 9.32/20056 bps Vs 15577 pps/r 9.35/18829 bps with over 400000 2-tuples. Ran this a second and third times and got similar results. I posted the results that favorites the most your code, which is still a loss of ~5%. It is better to start manually multiple β3.1 instances than using your current code.

Ratios should not be not affected, so it is even enough to only consider the 1-tuples/s metric (and in this specific case, it is a better metric).

Note that when running multiple instances of your code manually (each with Instances = 1) like I did with the current official β3.1, I get similar results to running with Instances = 2 or 4. The confusion may come from here, making you think that there were no performance loss while this actually seems to be the case, compared to the current β3.1.
rieMiner - Riecoin solo + pooled miner
Personal Riecoin page (links, download,...)
freebitco.in - earn up to $200 in BTC each hour!

Rockhawk
Posts: 48
Joined: 29 Oct 2018, 21:12

Re: rieMiner - Solo + pooled Riecoin mining

Post by Rockhawk » 15 Nov 2018, 00:47

Ah ok - key piece of information is that the perf loss is in my new code regardless of whether multiple instances are being used! That explains why I wasn't seeing the problem when testing single vs multiple instances.

I'll take a look tomorrow.

Post Reply