Huge memory consumption with CACHE-EX

[galiano]

Here there is movement:
Today I wrote this
Several hours later there was new fix

Currently I have the opportunity to test, I'll do it tonight.

[CapNCooK]

Here there is movement:
Today I wrote this
Several hours later there was new fix

Currently I have the opportunity to test, I'll do it tonight.

Thank you, it seems that writing on the Oscam board is more effective than placing tickets into their Trac :P
I'm glad to see there is some attention for this problem now!

I tested both new revisions, both still having the rising-memory problem.
Latest release crashed within one minute on my cache-server.

I will put a message on Oscam's board also now.

Thanks for putting it under the attention!

[CapNCooK]

Update:

- Valgrinded latest build (7285)
- Updated ticket on Trac (http://streamboard.de.vu:8001/ticket/2676)
- Placed post at Oscam streamboard segfault thread (http://streamboard.de.vu/wbb2/thread...481#post422481)

[CapNCooK]

OK, there is some news.

It seems that the memleak fixes that Corsair has done a few weeks ago are effective.
The memleaking has stopped at that point, but memory is still "growing" by other causes.

I have a discussion with Corsair about this matter on another board, and it seems that Oscam's job queue (containing queued jobs from several components) is filled up by "CACHE-EX -> Send Cache" commands,
this appears to be why memory is growing.

--

The remaining question is: Why does this queue fill ?

Corsair explained that this could be due to the fact that peers have bad (home) connections, and cannot coop with the cache stream that Oscam is sending.
Oscam is putting these "jobs" into a queue, and causing the mem to fill endlessly.

You can check the size of the queue yourself, put Oscam into debug mode 1 from the status page, and filter the log on lines with "action 28".
You will see lines like this:

2012/08/27 01:38:56 1479E30 c add client job action 28 queue length 151406 ReaderA
2012/08/27 01:38:56 1479E30 c add client job action 28 queue length 156282 ReaderB
2012/08/27 01:38:56 1479E30 c add client job action 28 queue length 166223 ReaderA
2012/08/27 01:38:56 1479E30 c add client job action 28 queue length 166627 ReaderB
2012/08/27 01:38:56 1479E30 c add client job action 28 queue length 148690 ReaderA

The numbers in these lines represent the queue-size. (getting bigger, and bigger).

--

From what i know now, (and that's not enough to draw a final conclusion), the problem seems to be that this queue isn't cleaned-up when cache expires.
For instance, when you put max_cache_time=7 into oscam.conf, after 7 seconds cache is cleared from the available collection of cache in RAM, but not from the job-queue.

For those that wants to follow the disussion, its here: http://streamboard.de.vu/wbb2/thread...481#post422481
But remember this is a developers forum, containing technical posts only, so keep the overall discussion here please.

Thanks

[PlatoonRTV]

Anyone tested newest version: 7294?

[CapNCooK]

Corsair applied a dirty fix to prevent the job queue from filling, by dropping all jobs when the queue reaches 20 jobs.

This "fix" is really dirty, and i recommend not to use it!
On top, this fix let my Oscam crash within the minute, so the code is buggy also.

For now, if you want to exchange cache, use Oscam build <= 7190 with CC-Cam readers only in mode 3.

[turbopower]

I don`t met instability with 7190 and mixed camd35 + cccam. But i use uniq group for every new exchange

[CapNCooK]

I don`t met instability with 7190 and mixed camd35 + cccam. But i use uniq group for every new exchange

That can be so. But i do not warn for nothing

There cannot be loopbacks with CACHE-EX, they are eliminated by the de-duplicate system.

Put oscam in debug mode -d 512 and filter on "duplic", you will see:

2012/08/29 16:08:59 CEE400 c ignored duplicate pushed ECM 0100&00006A/0000/1415/00:0000 from user1
2012/08/29 16:08:59 CEE400 c ignored duplicate pushed ECM 1722&000000/0000/0204/00:0000 from user2
2012/08/29 16:08:59 CF5940 c ignored duplicate pushed ECM 0100&00006A/0000/51B8/00:0000 from user1
2012/08/29 16:08:59 CF5940 c ignored duplicate pushed ECM 0963&000000/0000/107D/00:0000 from user3
2012/08/29 16:08:59 C2FF10 c ignored duplicate pushed ECM 0100&00006A/0000/51B8/00:0000 from user2

So duplicate ECM's are simply not beeing sent to the job-queue, making loopbacks impossible.
Any rumors, that a complex group system has to be applied to prevent loopback are simply not correct.

--

Furthermore, i found out that exchanging cache with camd35 causes these errors:

2012/08/29 16:10:59 A42B20 p dropping client data for camd_reader1 11s
2012/08/29 16:10:59 A42B20 p dropping client data for camd_reader1 20s
2012/08/29 16:10:59 A42B20 p dropping client data for camd_reader1 30s

If you have many or less of these lines depends on your load.
At some point, (having about 40 camd35 readers active), my oscam was starting to crash.
In Oscam's trac, searching on this error, you will find out that this line is caused by several bugs, only existing in the camd35 (reader) code.

Now that i switched all my readers to cc-cam on 7190, i have a clean log, and my job-queue never exceeds 1 job.

[wimpie007]

"2012/08/29 16:10:59 A42B20 p dropping client data for camd_reader1 11s"
i saw those errors ALSO with CCcam clients, although more often with camd35 clients.
since i use :Client max idle:0
i DIDNT see those errors again!
do you use client max idle 0?
regards,

[CapNCooK]

do you use client max idle 0?

Yes, i have that in oscam.conf.