Cache-EX Loopback

[CapNCooK]

Sorry to post against anything you add here lately galiano, also no personal offense intended, but setting a max_time lower as the cycletime, will cause cache te be send twice (or even more).

To explain: cache comes in.. Is stored unique into table, and forwarded to peers. Then its deleted from memory after 3 seconds since you advise max_time = 0. If that same CW is beeing send again by another CE peer to your oscam, it will be send AGAIN to your peers, since it was dropped before, and therefore considered "new", again.

So yes, size is lower, obviously, since you dump all your cache out after the minimal 3 seconds, but traffic will rise!
In general, code-wise, "size" has (or should have) _no_ relation to traffic.

--

In short, and again without any personal offense, i dont have the feeling many people here do really understand and have the knowledge to judge about how CE works internally. Much advice here is pointing people into wrong thoughts/directions, and does not match how things were intended in code.

Trust me, i spend numerous long nights with blueven analysing how cache-ex works internally in SVN, and on many points.. its far from beeing perfect for larger CE setups. Therefore, in silence, there is beeing worked on cache-ex improvements for over 1,5 months now, which basically led to a complete rewrite of CE. (all code credits to blueven).

This new code is not yet released, and will not be released until completely stable, ETA is: ASAP, so please dont ask, it will not be released just now. However, among the long list of things already changed, im browsing forums to gather "new", and "overlooked" bugs which might need solving.

Small bugs that are still in SVN can/will be solved on request, but larger things like "High CPU" issues are more related to the way CE was designed in SVN. Since this "base" is already completely rewritten in current development, its impossible to "fix" in SVN.

Therefore, i would rather investigate against how CE was intended code-wise, against latest SVN, because doing discussions like we do now, leads to nothing but bad advise without permanent fixes. Using an older build is offcourse your own choice, but dont expect any fixes/advise, because really, 8755 is long past gone.

[ametovic]

To ametovic and/or galiano, can you please check;

- Size change between 8755 and 8894
- Size change between 8755 and 8895

I want to make sure 8895 is causing the higher size.. If so, it is correct behaviour since max_count is removed in 8895.

Hi,

yes, difference is definitely between 8894 and 8895, I get a jump from 4k to about 11k, note that in 8894 I had max_count set to 7500, it was nowhere near that limit

if you need more testing/checking do let me know

amet

[CapNCooK]

Hi,

yes, difference is definitely between 8894 and 8895, I get a jump from 4k to about 11k, note that in 8894 I had max_count set to 7500, it was nowhere near that limit

if you need more testing/checking do let me know

amet

Thanks for testing!
Are you mayb able to compare 8755 with 8894 ?

[ametovic]

Thanks for testing!
Are you mayb able to compare 8755 with 8894 ?

It feels/seems close ... I would say it's the same

[CapNCooK]

It feels/seems close ... I would say it's the same

OK - Well, its what i would expect, seeing the codechanges from 8755 to 8894.

--

8895 removes max_count for a lot of reasons. This means you have to control cachesize with a combination of correct max_time, clienttimeout, CAID filtering and cacheex_maxhops.
I can tell you, pushing cache _eats_ CPU in all SVN, so rather than sending the full shebang to all your peers, use a limited/filtered part of your size to send out.
Also, use one group for incoming Cache-EX in your oscam, to process de-duplication correctly. Avoid sizes larger than around 12.000. Optimally, keep it somewhere below 10.000 for optimal hits.

If you need help, let me know.

[ametovic]

OK - Well, its what i would expect, seeing the codechanges from 8755 to 8894.

--

8895 removes max_count for a lot of reasons. This means you have to control cachesize with a combination of correct max_time, clienttimeout, CAID filtering and cacheex_maxhops.
I can tell you, pushing cache _eats_ CPU in all SVN, so rather than sending the full shebang to all your peers, use a limited/filtered part of your size to send out.
Also, use one group for incoming Cache-EX in your oscam, to process de-duplication correctly. Avoid sizes larger than around 12.000. Optimally, keep it somewhere below 10.000 for optimal hits.

If you need help, let me know.

I am no expert but looks that there is more to it, 8894 only has 4k while 8895 has 11k in cache. Max_count was set to 7500 and I was not hitting the limit, change to 8895 and bang, we have 2.5 times as much... I don't mind the size increase, I just don't see the % up in hits.

i am on 8755 now and it's reasonable CPU/bandwidth with same hits. maybe I'll stay there until new changes hit the SVN... unless you have more suggestions ? I can test, build and do C a little , so if anything is needed let me know

[CapNCooK]

I am no expert but looks that there is more to it, 8894 only has 4k while 8895 has 11k in cache. Max_count was set to 7500 and I was not hitting the limit, change to 8895 and bang, we have 2.5 times as much... I don't mind the size increase, I just don't see the % up in hits.

i am on 8755 now and it's reasonable CPU/bandwidth with same hits. maybe I'll stay there until new changes hit the SVN... unless you have more suggestions ? I can test, build and do C a little , so if anything is needed let me know

Yea, probably there is more to it.. 8895 was a _big_ update (see release notes here: http://www.streamboard.tv/wbb2/threa...605#post477605), which also introduced some new bugs (like cacheex loopback, a CWC compatibility problem and more).
The things known were fixed in later builds.

8755.. well, its upto your own choice. But looking at the timeline and changes fro 8755 to latest svn..
Lots, and lots of things are fixed and changed.

Did you ever try to use latest SVN, and adjust settings like i advised earlier ? (to get a decent cpu usage, size and hits ?)

[galiano]

Last night, after reading the posts here, I tried about 15 different versions and settings. Load of all new builds is almost equally high, this made ​​me to get back to 8755. With this version I get the best results with the lowest load, respectively smallest size cacahe.
I followed carefully parameter max_time, like you said. It turned out that even to set max_time = 0, oscam added timeout by default, I think it was 2-3 sec., so that it not can be set to "0". Furthermore, 8755, there is no problem with memory leak , in contrast to newer versions.

[CapNCooK]

Last night, after reading the posts here, I tried about 15 different versions and settings. Load of all new builds is almost equally high, this made ​​me to get back to 8755. With this version I get the best results with the lowest load, respectively smallest size cacahe.
I followed carefully parameter max_time, like you said. It turned out that even to set max_time = 0, oscam added timeout by default, I think it was 2-3 sec., so that it not can be set to "0". Furthermore, 8755, there is no problem with memory leak , in contrast to newer versions.

"Load" is directly linked to "Size" in SVN. So higher size == is higher load.

Indeed you cant set 0 for max_time, (you cant store cache 0 seconds), so clienttimeout + 3 seconds is the minimum value.

What i would like to know is, when you use a latest build, and trim max_time + cacheex_maxhop in a way it gets about the same size as 8755 => is CPU load different ?

Memleak, yea.. small in SVN.. but there.

[galiano]

max_time is same in both cases
8755 and cacheex_maxhop = 3, Size is 3000/4500
latest builds and cacheex_maxhop = 2 + cacheex_drop_csp = 1, Size is 10 000/12 000 (This situation puts me with a choice: either to switch to an older version or delete some peers, I chose the first)