Child pages
  • [caching] Ways to rebuild category cache
Skip to end of metadata
Go to start of metadata

Imported From: http://groups.google.com/group/in-portal-dev/browse_thread/thread/20883ec3e6b4be05#

In-Portal rebuilds category cache after a change has been made to a category.

Right now cache rebuild is always made. Following rules apply after category has been changed:

  • if user has less then 30 categories, then cache rebuild is made automatically using progress bar
  • if user has 30 and more categories, then user is asked if he wishes to perform cache rebuild right now using progress bar and he can deny rebuild
  • if "quick category cache rebuild" setting is enabled, then category cache is rebuild in background without asking a user or showing a rebuild progress bar

On one of my projects I saw a customization, when new setting was added "Automatically Rebuild Category Cache" was added and it was disabled. This was done, because there were 3000 categories and they were changed very often, so answering No all the time to "rebuild category cache now" question became very annoying.

I'm proposing to create new configuration setting "Category Cache Rebuild Mode", that will replace "quick category cache rebuild" option with following options:

  • Manual - user need to click on rebuild button in catalog (will use progress bar)
  • Silent - cache will be rebuild in background (no progress bar)
  • Automatic - ask/don't ask user (based on category count) and rebuild (if he agrees) with progress bar

Default setting will be set to Automatic for backwards compatibility.

Related Tasks

INP-1081 - Getting issue details... STATUS

18 Comments

  1. I'm 100% for it! I had many problems with users not rebuilding cache. and I even see no benefit to have Automatic or Manual mode, if it can performed silently in background, without annoying user, why don't do it every time it's needed?

    Envoy

  2. Phil, I guess you never worked with 3000 categories project :)

    This is where it's good to have these options.

    DA

  3. Dmitry, that's right, I don't have websites with so many categories.

    The main website I'm working on, as webmaster, have 390 categories, and 10 languages. Let me tell you that at this point, it's even impossible to edit 5 categories at the same time, otherwise everything explode when I try to save changes! Not to say that Debug mode need to eat more than 50mb (!) to finish its output, giving the need to change mysql config, just to be able to work correctly... If you are interested by categories bug, I can provide you some recipes, even with my small websites :)

    And categories "not appearing after creating" is a frequent bug, even on standard install with 10 new categories added. And until pressing "Refresh", no chance to view them (nor recipe here sadly).

    That's why I propose to do it silently. Every time. For every sizes.  

    Envoy

  4. We introduced progress bar specially to prevent memory leak on silent
    category cache rebuild.

    Not using that progress bar could cause memory leaks.

  5. to deep into my example, memory leak happen before categories are saved (temporary tables not closed in DB).
    Anyway, I think that rebuild categories as frequently as possible, and in a background process (that user couldn't stop), his something important.

    What are the pro and cons of a background process vs. progress bar?  

    Envoy

  6. Until cache is rebuild you don't see new categories you've added.

  7. Then cache rebuild should happen silently on every category creation, isn't it? +1 for silent cache rebuild :)

    Envoy

  8. And you'll get 100% memory leak. That's why I'm against that.

  9. I support proposed idea:

    Create new configuration setting "Category Cache Rebuild Mode", that will
    replace "quick category cache rebuild" option with following options:

       - Manual - user need to click on rebuild button in catalog (will use
       progress bar)
       - Silent - cache will be rebuild in background (no progress bar)
       - Automatic - ask/don't ask user (based on category count) and rebuild
       (if he agrees) with progress bar

    Default setting will be set to Automatic for backwards compatibility.

    *The only question that I have to Alex* - what are the options of updating
    silently ONLY current category level so it does show up or it will break
    cache date integrity in some ugly way?

    DA

  10. There was a code before, that only updated categories in path of changed
    category (this and all it's parents), but this code stopped working, when
    we introduced TreeLeft and TreeRight columns in category to speed up
    detection of all category children recursively.

    Basically "TreeRight - TreeLeft" = "sub-category count on all levels", but
    when only direct parents (instead of whole catalog) was rebuild we got into
    situation, when these numbers were incorrect, because they only counted
    these direct parents and not all category children.

    I'm not sure if there are users, who restrict (using CATEGORY.VIEW
    permissions) what categories are visible to what admin users, but they are
    none, then we can always show all categories in admin console (without VIEW
    permission check) and rebuild category cache in background in cron. Since
    we won't be checking VIEW permission (which is retrieved from cache) admins
    will see new categories instantly (maybe with incorrect sub-category
    counters) and cache would rebuild in background. Front-end users will see
    old category structure, since they still would rely on category VIEW
    permission checking.

  11. I think we should have CATEGORY.VIEW permissions and perhaps other
    permissions NOT checked/used by default.

    What's the need of this to users when it's really used (80-90% not used).
    We can make default setup somewhat lightweight and allow to enable full
    Permissions via System Settings or something.

    What do you think?

  12. But if users do enable them, then we're back as square one. We still need
    to decide what to do, when these permissions are enabled.

    On Fri, May 4, 2012 at 8:25 PM, Dmitry A. <dandre...@gmail.com> wrote:
    > I think we should have CATEGORY.VIEW permissions and perhaps other
    > permissions NOT checked/used by default.

    > What's the need of this to users when it's really used (80-90% not used).
    > We can make default setup somewhat lightweight and allow to enable full
    > Permissions via System Settings or something.

    > What do you think?

  13. +1

    Le vendredi 4 mai 2012

  14. Funny, just today we came across similar issue in a one project where
    Category Import on Front-end didn't show new categories in Front-end since
    Category Cache Rebuild didn't run (at it would in Admin).

    Anyway, I believe we should proceed the following way:

    Create new configuration setting "*Section Cache Rebuild Mode*", that will
    replace "quick category cache rebuild" option with following options:

       - Manual - user need to click on rebuild button in catalog (will use
       progress bar)
       - Silent - cache will be rebuild in background (no progress bar)
       - Automatic - ask/don't ask user (based on category count) and rebuild
       (if he agrees) with progress bar

    *Default setting* will be set to Automatic for backwards compatibility.

    In the future, if we create ability to DISABLE Category.View permissions in
    Admin we can make Automatic option to behave accordingly (so it's not even
    triggered or similar).

    Agreed?

    DA

  15. So what's your take on this?

    DA

  16. Agreed. Just need to create task.

    Also to speed up cache rebuild we need to change the way how it is rebuild.
    For example right now we dig into category structure very much and that's
    why it's impossible to split that process in sub-processes that will
    complete task faster.

  17. Here is the task

    INP-1081 - Getting issue details... STATUS

    .
  18. Here is the patch, ready for testing.