SmartCJ Wiki

User Tools

Site Tools

Trace: • new_rotation_hints • import_replacements • new_rotation_faq
new_rotation_faq

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
new_rotation_faq [2020/03/11 05:15] – [How to add a big list of thumbs] adminnew_rotation_faq [2026/04/20 13:54] (current) – external edit 127.0.0.1
Line 3: Line 3:
 ===== What is mod_rewrite ===== ===== What is mod_rewrite =====
  
-mod_rewrite - That's an addon that helps to create nice URLs that do not exist on server. There's a mod for nginx and lighttpd as well.+mod_rewrite is a module for Apache that allows you to make "beautiful" URLs. The same module is available for both nginx and lighttpd.
  
-Let's let by default you have urls like http://domain/gallery/cool-gallery/index.html, in fact there's no such path /gallery/cool-gallery/index.html on server. Mod_rewrite transforms it into http://domain/scj/tube/?slug=cool-gallery , "nice" URL is for SE and users.+For example, default template URLs look like http://domain/gallery/cool-gallery/index.html, even though no such physical path exists on the server. The rewrite rule converts that URL into something like http://domain/scj/tube/?slug=cool-gallery. The human-friendly URL is there for search engines and users.
  
-.htaccess is a file that sets a list of these rewrite urls for apache. +The .htaccess file created during setup contains these rewrite rules.
  
 +A simple explanation of how it works:
  
-I'll try to explain how it works:+As mentioned above, /gallery/cool-gallery/index.html does not really exist on the server. To open that gallery directly, the real URL would have to be something like /scj/cgi/out.php?url=content&slug=cool-gallery, which is not very clean. So we add a rewrite rule to .htaccess that tells Apache how to transform one URL into another. In this case the rule is:
  
-As I said before there's such path as /gallery/cool-gallery/index.html on server. You have to use an url like /scj/cgi/out.php?url=content&slug=cool-gallery to see a gallery, and this url doesn't look nice. That's why we add a special rule into .htaccess that says how to transform urls +  RewriteRule ^gallery/(.*)/index.html$ /scj/cgi/out.php?url=content&slug=$1 
 +   
 +This means: if the URL starts with gallery/, then has some value captured by (.*), and ends with /index.html, it matches this rule. Each captured part gets its own number: the first one is $1, the second would be $2, and so on.
  
 +In this example, $1 becomes cool-gallery. Apache therefore rewrites the URL to /scj/cgi/out.php?url=content&slug=cool-gallery and the gallery is shown.
  
-RewriteRule ^gallery/(.*)/index.html$ /scj/cgi/out.php?url=content&slug=$1+URLs like /gallery/cool-gallery/index.html appear on pages because subtemplates contain markup like this:
  
-That means if URL starts with  gallery/, then we have "something" (.*) upto "/" symbol, and then it ends with "/index.html" - we have to transform it into /scj/cgi/out.php?url=content&slug=$1 where $1 is something that was in (.*)+  href="/gallery/<!--GALLERY_SLUG-->/index.html 
 +   
 +So if you want to change the URL format on the site, you need to change both the subtemplate and the rewrite rule so they match each other.
  
-Inn this example $1 = "cool-gallery". So Apache will transform it into  /scj/cgi/out.php?url=content&slug=cool-gallery and you'll see a gallery.+===== Why in Rotation - Groups does it show 10 gallery in my group, but on the site page or Rotation - List thumbs 20 =====
  
-You see urls like /gallery/cool-gallery/index.html at your site because you have something like href="/gallery/<!--GALLERY_SLUG-->/index.html in your templates+One gallery can belong to more than one group (category). You can see this when editing any thumb: it has one main group and multiple extra groups. Rotation - Groups counts only by the main group, so you can clearly see how many unique galleries there actually are.
  
-That means that if you'd like to change URLs - you need to change rewrites.+For example, we have 2 groups A and B. and 2 galleries - one has main group A, the other has main group B. But both have A and B marked in the ext groups field. 
 +Rotation - Stats will show that we specifically have 2 galleries, one in group A and one in group B.
  
 +On the site, in each of the groups you will see 2 galleries.
  
-===== Rotation - Groups shows that I have 10 galleries while Rotation - List thumbs - 20 ===== 
  
-Each gallery can be assigned to more then 1 group. You can notice it while gallery edit: there are 2 select fields for groups - main and ext. Rotation - groups counts using main group so we can see real amount of galleries in DB, while Rotation - list thumbs shows galleries using Ext groups , the way you'll see it on site.+===== Custom gallery with my local content =====
  
 +- Upload anywhere thumb
 +  - when importing, specify Image List = listing of URLs of images
 +===== What is Init cats in Rotation - Special or why I added a category in the admin panel, but it didn’t appear on the site =====
 +
 +Recalculating categories, namely choosing the best category thumb and counting how many thumbs there are in a category, is a rather difficult task for the server, so it does it once every 15 minutes, stores the result in cache and all other parts of the script use cache. Init cats is a forced launch of this operation.
 +
 +If you have added a new category, it will appear in a maximum of 15 minutes, if you need it faster, click this button.
 +
 +===== What is cache  =====
 +
 +The script does not generate static HTML pages (see the question about mod_rewrite above), all pages are dynamic. In order not to recreate the page for each user, the script creates it once and saves it in cache for a certain time (CACHE_TIME), the next visitor during the lifetime of the cache will receive this page from the cache, without loading the server.
 +
 +The cache time is set in common.php CACHE_TIME, starting from update 46, the time in common.php can not be specified, but can be specified in the rotation settings. Please note that the time specified in common.php takes precedence; if specified in common.php, the script ignores the rotation settings.
 +
 +In order to view a page without cache, you need to add the skip_cache=true parameter to the URL. For example, http://domain.com/?skip_cache=true
 +
 +Starting with update 46, recreate cookie appeared - Rotation - Special - Recreate visited pages - the script places a secret cookie in your browser for 10 hours (you can remove the cookie at any time there). If you access any page of your site with this cookie, this page will be recreated (the cache for this particular page will be reset). It can be useful when changing the design of one specific page.
 +
 +The difference between skip_cache and recreate cookie is that skip_cache will show the page without cache only to you and that’s it, and recreate cookie will show the page without cache, then it will be put in the cache as you see it and other users will see it from the cache in this form.
 +
 +Once again: the page was created at 00 minutes. Cache time 10 minutes. in 05 you change the design.
 +if you looked at it in 05 with skip_cache, you will see a new design, but a user who came in 06 (before the cache expires) will see the old design.
 +if you open a page with a recreate cookie in 05, then the user who came in 06 will see the new design.
 +
 +
 +===== Cache Engines (New) =====
 +
 +The basic principle of the engine is to generate the desired page, put it in the cache for a certain time and display it until the cache is cleared. What engines are currently available:
 +
 +**File** 
 +Basically cache is stored in files (scj/cache - file cache). 
 +Pros - it works everywhere and at once. 
 +The disadvantages are that it is a file cache, it is not as fast, the cache system is not as efficient as we would like, and you have to separately take care of the size of the cache folder.
 +
 +**Memcached**
 +At one time, this was certainly a big step forward in terms of cache.
 +Pros - everyone knows it, admins install it on hosting without any problems, it’s easy to administer, etc.
 +Disadvantages - it is almost impossible to get any understanding of the actual memory occupied; when the server is overloaded, the entire cache is deleted and you need to “warm up the cache” to put the most used data there. When the server is overloaded due to load, this is a serious problem - the server is already loaded, and then all the cache has disappeared.
 +
 +All data is cached equally - it doesn’t matter if it is the most used or used once an hour - they all hang in the same memory and the service does not know how to dump them to disk, for example.
 +
 +With memcache, you cannot clear cache for one site only. All sites share the same cache space, so clearing it affects every site at once. You can of course run multiple memcache instances on different ports, but that usually requires admin help and is inconvenient in practice.
 +
 +How to install:
 +
 +- ask the admin to install Memcached
 +  - register memcache in the config 
 +<code>
 +$config['memcached_host'] = '127.0.0.1';
 +$config['memcached_port'] = '11211';
 +</code>
 +
 +
 +
 +**Therefore, now there are new engines that are recommended for use**
 +
 +In new engines, which are basically called NoSQL solutions, there are many more possibilities than will be described below, but for current cache purposes the basic ones will be described.
 +
 +The easiest solution to switch to is **Couchbase** 
 +**CouchBase** is a continuation of MemBase, apparently the closest descendant of memcachedb, which in turn comes from memcached.
 +
 +The main thing is that practically no movements are required from you in order to start using it. You install CouchBase and it works exactly the same as memcache. In order to start using it you need 
 +  - ask the admin to install CouchBase
 +  - add extension couchbase to PHP
 +  - register in config.php
 +<code>
 + $config['couchbase_host'] = '127.0.0.1';
 + $config['couchbase_port'] = '11210';
 + $config['couchbase_username'] = 'testuser';
 + $config['couchbase_password'] = 'testuser';
 + $config['couchbase_bucket_name'] = 'test';
 +</code>
 +
 +and that's it, you are already using a modern version of memcache. You can use this even with version 48-49.
 +
 +**How is this better than memcached** 
 +  * There is no concept of "warming up cache". In fact, this is a file cache, but made an order of magnitude more correctly, which loads the disk much less, calculates when and how best to write data in blocks to disk, etc. In this case, the data lies in several files, and not in hundreds of files like a file cache.
 +  * Parsed data disappears from the disk/memory on its own; you don’t have to worry about it yourself.
 +  * You can, just like with memcache, limit the amount of memory allocated for cache, but cache will not be limited by this amount. Even if there is more data, it will leave the most active ones in memory and download the rest to disk. And loading the required element from disk is faster than generating it again.
 +  * You can quickly and easily scale the system from the web admin panel.
 +  * You can also create several separate caches. In Couchbase this is like running several memcache instances, but everything is managed from the web admin panel, which is much faster and more convenient. For example, you can create 10 caches, each on its own port, and assign a separate port to each site. In that setup, the long-requested “Delete all cache” button effectively becomes possible, because you can clear the full cache for one site only.
 +  * When moving, cache is easy to copy because it is not a lot of small files that are copied, as with a file cache, but one database.
 +
 +
 +**Redis** is a separate NoSQL database project written from scratch. Its meaning for our purposes is almost the same (in reality, couchBase is more of a document oriented storage, and Redis is a purely key-value storage). To work with it, functionality was added in version 50.
 +
 +In order to start using it you need 
 +  - ask the admin to install Redis 
 +  - ask him to install the redis module for PHP 
 +  - register in config.php 
 +<code>
 +$config['redis_host'] = '127.0.0.1';
 +$config['redis_port'] = '6379';
 +$config['redis_database'] = 0; 
 +$config['redis_password'] = '';
 +
 +if redis is hanging on a socket it will look something like this
 +
 +$config['redis_host'] = '/tmp/redis.sock';
 +$config['redis_port'] = '0';
 +
 +
 +</code>
 +
 +and that's all.
 +
 +In this config, in addition to the clear fields, there are several new ones. redis_password - in most cases it is not relevant since now everything is on dedicated servers. 
 +redis_database - Redis does not have alphabetic database names, but rather numbers. If you have several sites, you can give each site its own number. But you don’t have to specify it and use one database for all sites. The difference will be that if you want to reset the entire cache and you use one database for all sites, the cache will be deleted for all sites. If each site has its own database, the cache will be chipped in for one.
 +
 +**How is this better than memcached** 
 +  * All the same advantages apply to couchbase. 
 +  * Redis is a more cache-specific solution, but CouchBase has a ready-made convenient admin panel (talking about version 2+).
 +
 +
 +
 +===== Where is cache stored? (memcache) =====
 +
 +By default, a file cache is used (stored in /scj/cache), but you can also use, for example, Memcache. To do this, you just need to add it to scj/includes/config.php
 +
 +  $config['memcached_host'] = 'localhost';
 +  $config['memcached_port'] = '11211';
 +
 +The address and port may change (check with your admin), but in most cases they will be the same.
 +
 +It makes sense to configure memcache to work via socket - it's faster. In this case, the config will be something like this
 +  
 +  $config["memcached_host"] = "unix:///tmp/memcache.sock";
 +  $config["memcached_port"] = 0;
  
-===== Where pages are stored ===== 
  
-With New Rotation you get a lot of pages. Old style rotation was like you create a finite number of pages (usually up to ten)+There are several nuances: 
 +  - file cache is not as slow as it may seem. In practice, the difference is often smaller than people expect. 
 +  - with file cache, the cache stays on disk. If the server is overloaded or restarted, the cache does not simply disappear and the script does not have to rebuild everything from scratch. With memcache, the entire cache can be lost, and then the script must regenerate many pages on the fly, which creates extra load again. If you do use memcache, make sure it has enough memory. The exact amount depends on database size and on how heavy your templates are, since the cache stores the generated HTML itself. As soon as memory runs out, pages start being regenerated on the fly. Memcache size also matters because old entries are evicted when space is needed. For example, if the cache can hold 10 one-megabyte entries, adding an eleventh entry pushes the oldest one out. Another issue is fragmentation: memcache stores data in blocks, so real memory usage is higher than the reported payload size. Roughly speaking, if a block is 100 bytes and you save 1 byte, statistics may report 1 byte while 100 bytes are actually occupied. If an entry is evicted, memcache does not report an error back to the script, so the script cannot react to that directly. A rough practical estimate is about 15% fragmentation.
  
-Pretty common situation: 300 categories, 7 pages in each, 3 ways to sort galleries inside category (CTR, date, duration) - it's 6300 pages.+===== Why in my admin panel and on such and such a page of the site the thumbs are in a different order =====
  
-It doesn't make sense to create those pages each minute so script creates it when someone requests it, stores in cache and uses cached version until it expires.+With the same sorting, the order should be the same, taking into account the following nuances:
  
-Thus we have 2 points to remember:+1. in the admin panel it shows the name as it is in the database according to CTR, for example, on the site page in some places there may be “new thumbs” that are being tested. For example, on the page thumb 123 is in 15th place, and in the admin panel it is in 50th place. This is probably a new thumb that the script is testing.
  
-1. It takes time to create pages. if cache is empty it is called 'cold startup', we have to "warm up" cache , ie create some mostly visited pages.+2. The new thumb may get an unexpectedly high CTR. For example, they showed it 2 times and clicked on it 1 time = CTR 0.5, but 2 impressions are not a reason to judge that the thumb is good. Therefore, by default, “new” thumbs are not shown in first place. You can change this in settings (New Rotation: New thumbs placement : Place new thumb (shows less then New thumbs timelive) on places other then Test positions start if it has good CTR. ). It’s easy to check if you select New Thumbs = dont show new in List Thumbs.
  
-2. Once cache is up it doesn't really matters how many visitors you have as it almost does not consume any resources to give away a page from cache.  
  
 +===== Why is there an empty tag cloud on my page =====
  
-Thus if you have 1 site with  50k or 500k traffic - it doesn't affect your server load too much as pages in most case will be cached. But if you have 5 sites with 10k traffic on each - it won't be the same at 1 site with 50k traffic as we have to create a full set of pages for each site.+A tag cloud is generated from active tags every 30 minutes and stored in cache. Accordingly the options:
  
 +* no active tags
 +  * less than 30 minutes have passed
 +  * you nailed cache
  
-Beside that your layout affects load: ie if you have 3 options to sort content it will be "total amount of pages * 3", while if you have just 2 ways to soft it (it depends design actually , ether you have a link to another way to soft it or not) - you'll get less page to create => less load on server. 
  
-===== Thumbs (Galleries) are not getting deleted =====+===== Thumbs (galleries) are not deleted =====
  
-  * up to update 46 there was a problem with deletion of large parts of thumb, script was not able to delete lat's say 1000 thumb during 30 seconds browser timeout time. +* previously there was a problem when deleting a large number of thumbs. For example, if one sponsor had 10k thumbs and you deleted that sponsor, the script tried to remove everything immediately and could hit a timeout. The issue was even worse if you needed to remove not just thumbs, but a full custom gallery. 
-  * that's why since update 46 thumbs won't be actually deleted once you press 'Delete', instead those thumbs will be marked as 'Marked for deletion' and once every 10 minutes cron script will check for those thumbs and actually delete it. +  * now thumbs are deleted by cron. When you click “delete” in the admin panel, the thumb is not deleted instantly. It is moved to the 'Marked for deletion' status, and every 10 minutes cron checks for such items and removes them 
-  * you want to force that action you have to run following command in SSG+  * if you need to delete “right now” - run rotation.php with the parameter process_deleted=true 
   * <code>cd /PATH_TO_/scj/bin/; env HTTP_HOST=yourdomain.com php rotation.php process_deleted=true </code>   * <code>cd /PATH_TO_/scj/bin/; env HTTP_HOST=yourdomain.com php rotation.php process_deleted=true </code>
-  * there's one exclusion - if you delete thumbs in 'List Thumbs' and total amount of thumbs being deleted is less then 30.+  * the only exception: if you delete a thumb in Rotation - List thumbs and less than 30 thumbs are deleted, then the deletion occurs immediately.
  
  
-===== What is cache =====+The deletion speed is hard to predict and depends on the number of galleries involved, current server load, whether a custom gallery is involved, and whether the files are on the current server or on FTP.
  
-Script does not generate static page each minute or something like that. Instead it creates a page once someone request it and saves it to cache with expiration time = CACHE_TIME, so each next the same request will get this page from cache till expiration time.+===== How much traffic is needed to rotate thumbs =====
  
-Yo can set CACHE_TIME in Rotation - Settings , but if for some reason you want to overwrite it somewhere - you can add in common.php +It is not necessary to revise everything to the end for good sales, but in general it is very easy to calculate how much traffic you need.
  
-  if (!defined('CACHE_TIME')) define('CACHE_TIME', 900);+A thumb is considered rotated after it has been shown New thumbs timelive times, which is 500 by default. 
 +Let’s say there are 200 thumbs on the page, and the default percentage of test positions on the page is 15. That means new thumbs will be shown in 30 of those 200 positions. 
 +Every visit to the page means +30 impressions for new thumbs. 
 +For example, we have 10,000 thumbs, to rotate them we need 10,000*500 = 5,000,000 impressions. One visit to the page is 30 impressions, but to rotate the base to 10k we need 5M / 30 = 167k page loads.
  
 +===== Not every thumb of a gallery is tested =====
  
-If you want a page without cache - add &skip_cache=true to any request for example http://domain.com/?skip_cache=true. Note that this request will only show a page to you and will NOT save this page to cache.+Apart from the banal lack of traffic, it usually means that the gallery has, for example, 10 thumbs, but only one has impressions. How this happens: for example, we have 100 galleries of 10 thumbs, 20 thumbs on the page. Let's start rotation. All have a CTR of 0. We displayed the first thumb with gallery ID 1 - 20. They received a CTR greater than 0. We need to refresh the page. test ones for example 5 on the page.
  
-If you want script to recreate a page AND save it to cache - you have to go to Rotation - Special - Recreate visited pages and it will add a cookie to your browser so each page you visit will be recreated and saved to cache.+We display IDs 1 - 15 as the 15 best (and so far only the first thumb from the gallery has received CTR and impressions). And then 5 test ones - here we can take thumb number 2 with gallery ID 16-20
  
-For example: A page was created at 0 seconds. Cache time = 1000 seconds. +It turned out that these thumbs scored less than thumb 1 gallery ID 1-15, we take for example the 3rd thumb with gallery ID 16-20.
  
-At 200th second you change design. If you or any other surfer open that page at this time - you will not see new design.  
  
-If you open with 'skip_cache' - you'll see new design, surfers - not.+Thus, it turns out that gallery ID 1-15 has impressions only for the first thumb. When their CTR is less than the CTR of others, they will be “squeezed out” from the top.
  
-If you open with 'Recreate cookie' - you'll see new design and surfers will see new design.+They also have the opportunity to be shown on page 2 of the pagination, but again, provided that your traffic reaches the 2nd page.
  
  
-===== How much memory consumes cache =====+===== Parameter out.php &link= ... =====
  
-It 100% depends on your site.+Initially, out.php has a &link=blabla parameter, the meaning of which is that statistics on blabla can be seen Stats - Links. But rotation uses it to pass rotation parameters (12x123x1234). This looks very simple in practice: for example, the link /gallery/cool-desc/index.html?12x345x567 rewrite (see what it is above) is converted into a URL like out.php?slug=cool-desc&link=12x345x567 and later the script processes statistics on links, converting this into statistics on thumb. Accordingly, if you have a URL like out.php?member=domain.com&link=top, then this option still works the same, but it cannot be used for rotation links. In order to solve this situation, another parameter &link2 was introduced, the meaning of which is that you can do out.php?slug=cool-desc&link=12x345x567&link2=blabla and still view statistics on links and rotate the thumb at the same time.
  
-We cache ready-made HTML in cache. So let's say your index page is 100k.  
-Plus you have 100 categories, each category page 100k, and you have 50 pages within category , so that's 100 category * 50 pages * 100k = 500M 
  
-but a user can see those pages in 3 sort modes so 500 * 3 ..+===== What are thumb categories =====
  
-Also let's say you have 100 000 galleries , each page is also 100k. So that's 10gb of html.+A category thumb is the default thumb for a category. However, you can configure the script to use a thumb that is not the absolute best one, for example the 2nd or 3rd by CTR. These settings are available in Rotation - CMS - Settings.
  
-Of course, HTML gets compressed so it consumes less memory, but you can get an idea.+===== How to create a master site from a slave =====
  
-Usually when the cache is full an old record get deleted and a new one - stored in cache. Thus if a surfer requests an old one - we'll have to generate a page again rather then just pull it out of cache.+- Dump the master database 
 +  - We look in rot_linked_db for the number of the slave that we are disconnecting (X) 
 +  - Disconnect the desired slave from the master 
 +  - Fill the database with the former on the left (dump) from point 1, but only the tables rot_gallery_info, rot_gallery_statsX, rot_gallery_dataX and rot_thumbs 
 +  - rename rot_gallery_statsX to rot_gallery_stats1, rot_gallery_dataX to rot_gallery_data1
  
-===== Cache Engines (New) ===== 
  
-The main idea of cache is generate a page once and then show it to every visitor until cache expires. So we need a place to store cache data 
  
-**Regular Files**  
-Usually (bu default) we store cache in file (scj/cache - file cache).  
-Pros - works everywhere, no need to setup anything.  
-Cons - file cache means a lot of files on disk, and it's slow. 
  
-**Memcached**+===== Gallery statuses =====
  
-Pros: easy to setup, easy to manage. +Pool - Active - Old - actually necessary for Shift settings to work in rotation groups. The point is to fill the site automatically. So you added a lot of galleries to the pool status, set 10 galleries to be added once a day and the site adds them from the pool to active ones, and they appear on the site. This gives the appearance of daily updates. Old - if you want to remove old galleries from the site while keeping the total number the same.
-Cons: you lose all your cache data when server reboots or memcache is restarted. So each time after that you'll need some time to "warm up" cache ie create pages.+
  
-Also unless you have more then one instance of memcached - all your cache data will be stored within one instance and if you click at "clean cache" at one site , this will kill cache for all sites.+On the site, the listing shows only galleries with the active status.
  
 +preload - galleries are added to Preload (in the admin panel) so that you can choose which of the grabbed thumbs will eventually be on the site. Those that you do not select will be deleted.
  
-**That's why there are a couple of new cache engines**+to_grab - when galleries are added to the database they receive this status before the grabber creates a thumb for them. At the moment thumbs are created, they receive the processing status. to regrab - gallery is waiting for regrab.
  
-The easiest one to migrate to is **Couchbase**  +to delete - gallery is waiting to be deleted. Many galleries are marked for deletion, and it is impossible to delete them instantly, especially
-**CouchBase** - it's a successor of MemBase (inspired by memcached I believe)+
  
-The main advantage of it is that you don't have to do a lot to start using it. +grab error - if something went wrong while grabbing thumb, the gallery is either deleted or given the grab error status depending on your settings. 
-  - Ask admin to install couchbase + 
-  - add the same strings as for memcache+Banned - the URL remains in the database and will not be added again. Like grab error, this only makes sense if you are adding a gallery via import sets and there is a possibility that some galleries will be added again and again. For example, you add a set through import, gallery goes to preload. You selected part of the gallery, but decided not to add some. An hour later, this import set is added again, and there is the same gallery again - it will again end up in preload and you will have to sift it out again. To avoid that, you can add it to banned. The downside is that the database in this case grows and there is a gallery that is not used, but takes up space in the database. 
 + 
 +Inactive (the equivalent of hidden in version 1.X) means the gallery is not shown in listings, but is still available by direct URL. For example, imagine a gallery was indexed but hotlinked its content from somewhere else. Later the source content was deleted and the gallery stopped working. Previously, the checker could delete such galleries, or mark them inactive. The downside was that for CE spiders the old URL would now return 404. In some cases it is better to return a page without content instead of a 404. That is what the “Hidden” status is for: the gallery remains accessible by direct URL, but its thumb is removed from rotation. This status can be assigned by an import set that removes galleries. 
 + 
 +Gallery Checker has a similar option. 
 + 
 +The exact effect of this on the CE is unknown. 
 + 
 + 
 +===== Category statuses ===== 
 + 
 +Categories have 3 statuses: 
 + 
 +- active - displayed in <category and gallery from it are displayed in <thumb 
 +  - not active - not displayed in <category and gallery from it are not displayed in <thumb 
 +  - hidden - **not displayed** in <category, but gallery from it is displayed in <thumb 
 +===== How to import a very large dump ===== 
 + 
 + 
 +Version 2.* can import large files through importset on its own, in practice handling very large files as long as they are reachable by URL. Just add the URL and the script will gradually import everything. 
 + 
 + 
 +===== eval()'d code ===== 
 + 
 +If, when you access any page, you get something like “Parse error: some error, in /some/path/scj/tube/index.php(0) : eval()'d code on line 1717” - this 100% means that you have an error in your PHP code in the template. 
 + 
 +In my experience, the most common problem is missing quotes. http://smartcj.com/wiki/doku.php?id=new_rotation_hints#php_code_in_template 
 + 
 + 
 +===== Crop Profiles ===== 
 + 
 +These are sets of parameters for processing thumb. 
 + 
 +If you only need the size on one side, but set the second side to 0. For example, if the width is specified and the height is 0, then the width will be fixed, and the height will change to fit the thumb size. This is true for sites like Pinterest. 
 + 
 +Image Command is a command that is applied BEFORE the thumb is made. the only parameter {FILE} is the current image. For example, 
 + 
 +  /usr/bin/your_command -some_param {FILE} 
 + 
 +Thumb Command - similar only for the already made thumb. 
 + 
 + 
 +Face Detect - see [[FaceDetect]] 
 +===== How to grab custom galleries ===== 
 + 
 +Sometimes you may not want to create another slave site, but instead want to grab a custom gallery from one of your other sites, for example when the original URLs are no longer available. If the gallery is built as a page with pictures on a separate page, the only way to grab it is Deep fetch. The problem is that such pages often contain links to tags, categories, and other pages with many additional thumbs, so Deep fetch tries to download all of them. If the custom gallery also uses skimming, the whole grab can become very slow. 
 + 
 +It's better to do this: 
 +  - create a separate template, for example plain_gallery where all links will be directly to the picture, and not to a separate page 
 +  - Rotation - export will give you a URL like http://domain/scj/tube/?content_id=0a5cf5010394ea3a0111f3cce9cca0b7 
 +  - in any editor search/replace and you have http://domain/scj/tube/?force_template=plain_gallery&content_id=0a5cf5010394ea3a0111f3cce9cca0b7 
 +  - and now you give these URLs to the grab in another script 
 + 
 +===== How to copy rotation info ===== 
 + 
 +Rotation data is stored in rot_gallery_stats* tables, so if you need to copy rotation data from one slave to another you need 
 + 
 +- see what ID the slave has from which we are copying 
 +  - what is the slave ID to which we copy 
 +  - copy(!) the table conditionally rot_gallery_stats2 to rot_gallery_stats3 
 + 
 + 
 +===== Category tags does not work ===== 
 + 
 +The problem is usually in using a category tag on gallery pages, for example <!--CATEGORY_NAME-->, while the gallery may have more than one category and it is not clear which one should be displayed. Therefore, either a tag of the form <!--CATEGORY_1_NAME--> or the construction <groups ... >...</groups> is used 
 + 
 + 
 +===== Group Deactivation (Group Exclusion) ===== 
 + 
 +Let's say we have a master on which there are groups A and B. We connect a slave on which we deactivate group B. When sampling for a slave, we have 2 options: 
 + 
 +* selection of all gallery EXCEPT group B 
 +  * or all gallery from group A 
 + 
 +at first glance they are the same thing. However, if our gallery is in both groups, then in one case it will be included in the selection, but in the other it will not. 
 + 
 +So in settings there is a Group Exclusion setting 
 + 
 +- Hard = the selection goes like “all the gallery EXCEPT group B” (those galleries that are in both groups are not included in the selection) 
 +  - Soft = the selection follows the second option, namely "all galleries from group A" (those galleries fall into the selection) 
 + 
 + 
 +When is this needed? Imagine you have 3 groups: straight, guy, and teen. Suppose that on a slave site you want only straight content. In that case, a surfer interested in guy content should not see those galleries at all, so those groups become a strict exclusion. 
 + 
 + 
 + 
 +===== It does not grab this gallery  ===== 
 + 
 +Some pages may be designed in such a way that they cannot be raked automatically. 
 + 
 + 
 +In this case, you can make a tn gate to transform it into a more understandable form: 
 + 
 +* for example, URL gallery http://gallery/, thumb http://gallery/image.jpg 
 +  * we make file http://your_server/gate.php 
 +  * import dump from gallery to http://gallery/|http://your_server/gate.php?url=http://gallery/ (pattern url|thumb_url) 
 +  * your gate.php script receives a URL parameter and can pull out a picture from there that the smart will perceive as a regular picture and add to the database 
 + 
 + 
 +**Example:** 
 + 
 +Suppose we have a gallery at http://gallery/ with unusual markup:
  
 <code> <code>
-$config['memcached_host'] = '127.0.0.1'; +<html> 
-$config['memcached_port'] = '11211';+<body> 
 +<dont_grab_this src='http://gallery/0.jpg'></dont_grab_this> 
 +<some_strange_tag src='http://gallery/1.jpg'></some_strange_tag> 
 +<some_strange_tag src='http://gallery/2.jpg'></some_strange_tag>
 </code> </code>
  
-that's it. 
  
 +The standard script parser cannot parse this, so we make the gate http://my_server/gate.php which parses this gallery and shows thumb from there (the code is simplified for the example, of course more checks are needed)
  
-**Redis** - another great solution written from scratch.  
-  - ask admin to install redis 
-  - ask admin to install redis extension for php 
-  - add to includes/config.php 
 <code> <code>
-$config['redis_host'] = '127.0.0.1'; +<?php
-$config['redis_port'] = '6379'; +
-$config['redis_database'] = 0;  +
-$config['redis_password'] = '';+
  
-if your admin prefers to use sockets then +if (!isset($_GET['url']) or !$_GET['url']) die("Need url");
  
-$config['redis_host'] = '/tmp/redis.sock'; +if ($html = file_get_contents($_GET['url']) ) { 
-$config['redis_port'] = '0';+ preg_match_all("|<some_strange_tag src='(.*?)'>|", $html, $out);
  
 + $image_num = (isset($_GET['num'])) ? $_GET['num'] : 1;
 +
 + if ($out[1][$image_num-1]) {
 + header("Content-type: image/jpeg");
 + echo file_get_contents($out[1][$image_num-1]);
 + }
 +}
  
 </code> </code>
  
-That's it.+check that the gate is working
  
 +http://my_server/gate.php?url=http://gallery/
 +http://my_server/gate.php?url=http://gallery/&num=2
  
-===== How to add a big list of thumbs =====+At these URLs, the browser should display http://gallery/1.jpg and http://gallery/2.jpg from our gallery, respectively.
  
-Note, if you use version 2+ you can add big files as-is, no need to do any extra work.+Now import this gallery, pattern: url|thumb_url
  
-If you have really big list to import it could be really hard due to some reasons like size of POST is usually limited, server won't have enough time to process it and so on. Beside that adding a lot of galleries really fast may overload your server.+http://gallery/|http://my_server/gate.php?url=http://gallery/,http://my_server/gate.php?url=http://gallery/&num=2
  
-Good idea is load load this list smoothly using import sets. You have to +As you can see here, the URL will be http://gallery/, and for thumb we will take the images that are visible at the URL http://my_server/gate.php?url=http://gallery/ and http://my_server/gate.php?url=http://gallery/&num=2.
  
-  * create and upload file cut.php +Since we don’t know exactly how many pictures there will be in the gallery, we can add, for example, http://my_server/gate.php?url=http://gallery/&num=123 (that number is non-existent) to the thumb list - the script will simply not be able to grab it and will skip it.
  
-<code> 
  
-<?php+**Option 2: replacement at the import stage**
  
-error_reporting(E_ALL & ~E_NOTICE);+Let's say we have import set http://sponsor/dump.php which produces a dump of the form
  
-if (!$_GET['file']) die("\n You have to pass file=... parameter in URL"); +url|thumb_url|desc
-if (!file_exists($_GET['file'])) die("\n File {$_GET['file']} does NOT exists. Check path. "); +
-if (!is_writeable($_GET['file'])) die("\n File {$_GET['file']} is NOT writeable. Set 666 permissions. ");+
  
-if (!file_exists('./tmp.file') or !is_writeable('./tmp.file')) die("\n File tmp.file does not exists OR is NOT writeable. Set 666 permissions. ");+and we decided to make a custom gallery from this content, but the problem again is that the gallery has the same appearance as in the previous example, but the grabber cannot grab it in normal mode.
  
-echo passthru("head -n500 {$_GET['file']} "); +Making a gate - option 2: 
-if (!$_GET['test']) exec("tail -n +500 {$_GET['file']} > tmp.file; cp tmp.file {$_GET['file']} ");+  * gate.php?url=http://sponsor/dump.php 
 +  * the gate receives the feed URL, grabs it, processes each line as a gallery and adds a field with pictures for a custom gallery, which in this case the gate produces a dump immediately in the form of url|thumb_url|desc|**image_list** 
 +  * we add the set to the import immediately gate.php?url=http://sponsor/dump.php
  
-</code> 
  
-  * upload to the same location your dump and set permissions 666 +**Option 3: Import using XPath**
-  * Upload empty file tmp.file and also set permissions 666 +
-  * Add ImportSet with URL http://yourdomain/path_to/cut.php?file=DUMP_NAME&test=true +
-  * Click Test and select fields +
-  * cut.php outputs 500 lines and cuts dump by those 500 lines. This way you'll add everything as usual import set - gradually and smoothly.  +
-  * &test=true means do not cut dump by 500 lines this time. You have to remove this parameter after test. So your final URL will be http://yourdomain/path_to/cut.php?file=DUMP_NAME+
  
-That's it.+For example, we have a new tube that gives a URL like http://tube/gallery.html,; it doesn’t have an export; we grab flv_url and flv_thumb_url from there and make our own custom gallery from them.
  
 +* If you don’t know what xpath is, you should read it, self-education is useful
 +  * open your favorite browser - developer tools
 +  * open the page, find the desired element and do Copy - XPath
 +  * we get something like /html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2] , by now, if you read the first paragraph, you already understand that this is the path to the page element. This way the script will know where the element you need is located on the page. Of course, you don’t need the element itself, but its value, for example /html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href if it’s a link or @src if it’s an image, etc.
 +  * add url|flv_url|flt_thumb_url as a pattern, marking those fields in which you need to extract Xpath as XPath:/.... in our version it turns out something like
  
-===== Cache Engines (New) =====+  http://tube/gallery.html|XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href|XPath://html/head/meta[5]/@content 
 +   
 +If you insert this line into the import, the Test Xpath button will appear. If you click on it, the script will pull out the gallery URL and calculate the specified fields. This is necessary to check that you have entered xpath correctly. 
 +   
 +This way you can add as many URLs as you like
  
-If you read [[New Rotation FAQ]] you should know that script generates a page on request only. We do not generate million of pages by cron or something like that. When the page is generated we have to store if somewhere in cache. But we have different types of cache you can use.+<code>   
 +http://tubesite.com/gallery.html|XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href|XPath://html/head/meta[5]/@content 
 +http://tubesite.com/gallery222.html|XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href|XPath://html/head/meta[5]/@content 
 +http://tubesite.com/gallery333.html|XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href|XPath://html/head/meta[5]/@content 
 +</code> 
  
 +and thus the gallery will be added. But it's certainly not very pretty. Therefore, you can add Import Replacements where
  
-**File cache**  +  url contains = tubesite.com 
-A basic one. All cached pages are stored in files (scj/cache folder) +  replace FLV Url = XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href
-Pros - works everywhere, you don't have to do anything. +
-Cons - it's a file cache, it's slow, not so good cached by OS disk operation and script has to check all files for expiration date and so on.+
  
-**Memcached** +This way, you can simply import http://tubesite.com/gallery.html and the script will already replace the necessary fields for this site during import.
-it was a big step ahead some time ago.  +
-Pros: everybody know it. There's no proble for admin to setup it.  +
-Cons: sometimes it's hard to find out how much memory is actually still free, data gets fragmented, if you reboot server - you lose all cached data and need time to "warm up" cache after reboot. Memcache can not segregate cached data by popularity and stores everything in memory. It's really painful to reset cache just for 1 site for example.+
  
-**That's why there are some other cache engines you worth using**+As usual, the best approach is to add one gallery first and check the grab log.
  
-New cache engines (basically NoSQL dbs) deliver you more performance while consuming less CPU and HDD.+**Option 4: ahhhh, everything is complicated, I don’t want to read and understand anything**
  
-Easiest solution - **Couchbase** which was built on MemBase, based on memcachedb, which was inspired  memcached. It's really easy to start using it+If you don't want to do it yourself, you can always pay and have the work done for you. Any schoolchild can make a gate or copy xpath in a browser. If you don’t want to contact schoolchildren, contact the support with specific sites. Estimated price - $100 per site. 
 + 
 + 
 +===== Lazy load, load on scroll, pinterest and so on  ===== 
 + 
 +All these options are based on the fact that when scrolling to the end of the page, new content is loaded and the user does not have the page overloaded, but new content is added at the end of the page. 
 + 
 +It looks like this: imagine an index template with 50 thumbs
  
-  - ask admin to setup CouchBase 
-  - add 2 lines to scj/includes/config.php 
 <code> <code>
-$config['memcached_host'] = '127.0.0.1'; +<thumb num=1-50> 
-$config['memcached_port'] = '11211';+<a href='/...'><img src='<!--THUMB_URL-->'></a> 
 +</thumb>
 </code> </code>
  
-and that's it. You can use it with versions 48-49 too. 
  
-**Redis** - another nice key-value product . To you it you have to +We add, for example, jquery.wookmark to the template (not the only option, of course) as in http://demo.smartcj.com/scroll/ 
-  - ask admin to setup Redis  + 
-  - ask admin to add redis module for php +upon reaching the end of the page it must load additional content, for this we indicate from where 
-  - add following to scj/includes/config.php+ 
 +  var P_BASE = '/?force_template=index_scroll_ajax&page='; 
 +   
 +   
 +As you can see, the index_scroll_ajax template is loaded and &page= is specified. Each time when the end of the page is reached, jquery.wookmark will load this URL changing the page number 
 + 
 +Template index_scroll_ajax where we display 10 thumbs 
 <code> <code>
-$config['redis_host'] = '127.0.0.1'; +<thumb num=1-10> 
-$config['redis_port'] = '6379'; +<a href='/...'><img src='<!--THUMB_URL-->'></a> 
-$config['redis_database'] = 0;  +</thumb>
-$config['redis_password'] = '';+
 </code> </code>
  
-There are few new lines. redis_password - if you have a dedicated server most likely you don't want to setup it. redis_database - it's kinda number of slot in redis DB. Useful if you want to separate caches for your sites for some reason. 
  
  
-Any of those 2 new engines are better then memcache and definitely better then file cache. I'd recommend to use file cache in emergency case only.+===== Duplicate test thumbs  =====
  
 +The problem manifests itself on a database where thumb has not yet gained CTR\impressions at all: we open page 1, then page 2 (of the same sample) and we see repeated thumbs. The following happens:
  
-===== Custom Sitemaps =====+For example, imagine we have 3 thumb positions per page, and only 9 thumbs total. That means there are 3 pages. 
 +test position -1 per page, 1 test 2 rotated thumbs. 
 +On page 1: we take 2 thumbs by CTR, since all CTR 0 get thumbs 1 and 2. 
 +We take the first test one - 3. It turns out 1 2 3.
  
-Right after installation you should have a default sitemap template. Basically sitemap is a regular template, the only difference is that you have to output a special header to identify that it's a xml format.+Page 2 - we take the next 3 thumbs by CTR using the page offset, but since CTR values are still unstable, the sequence may become 4 5. Then we also need one test thumb, so the script again looks from the start, sees that thumb 1 still has no impressions and is still a valid test thumb, and produces 4 5 1. That is where the repetition comes from. 
 + 
 +As soon as the database gains a little CTR, it turns out that according to the CTR, a thumb is enough to type a page and thus the thumbs are not repeated. 
 + 
 +===== Fast Navigation  ===== 
 + 
 +If you have a large database, then it makes sense to replace <pagination> with just <!--PREV_PAGE--> <!--NEXT_PAGE--> 
 + 
 +The difference is that if we have navigation, we first need to calculate how many gallery there are for a specific request, this is a certain load on the database. 
 + 
 +If we only have links to the next/previous pages, then there is no need to count anything in the database. 
 + 
 +Please note that in this case the total_items tag will simply show the number of galleries on the current page. 
 + 
 +===== User Thumbs ===== 
 + 
 +Suppose your import contains many thumbs, but there is not enough traffic to rotate all of them, which is the usual case. Still, you may want to keep all of them for rolling thumbs. That is what the User Thumbs option is for: the specified thumbs are downloaded, the crop profile is applied, but they are not added to the database. They are stored only on disk in /scj/thumbs/user_thumbs/. 
 + 
 +Those we can import as URL|THUMB_URL|USER_THUMBS 
 + 
 +If it is not possible to provide two separate thumb fields in the import, you can simply import as URL|USER_THUMBS and set How Many Thumbs from each gallery ?= X. In that case, all thumbs from this field are saved to disk, while X thumbs are added to the database for rotation. 
 + 
 +This way the database does not grow and there is no need to rotate extra thumbs. 
 + 
 +At the same time, you need to understand that the script does not know anything about thumbs, which are stored in user_thumbs, so for rolling thumbs you have 2 options. Either for each gallery the number of thumbs should be the same, or when rolling, the script should preload thumb for rolling and thus understand how many there are in total. 
 + 
 +In the template, the URL to the directory with the user thumb for this gallery can be obtained with the tag 
 + 
 +  <!--USER_THUMBS_FOLDER--> 
 +   
 +   
 +   
 +===== Filter request parameters ===== 
 + 
 +For example, you have a website in English and there is a search form where people enter search queries on the site. A log of search queries is displayed somewhere on the site. Some people try to search, for example, in Russian and this is displayed in the query logs, but you would not want this to be visible. You can filter unwanted characters at the request stage. 
 + 
 +Add to common.php 
 + 
 + 
 +  if (isset($_GET['search'])) $_GET['search'] = preg_replace('|[^a-z,0-9,\s]|is','',$_GET['search']); 
 +   
 +   
 +and these will only be Latin characters 
 + 
 +There is also an option "Log empty search queries" and if nothing is found according to the request, it will not be logged. 
 + 
 + 
 +After the update, an option was added to the rotation settings that does the same thing
  
 <code> <code>
-<? echo '<?xml version="1.0" encoding="UTF-8"?>' ?>+Search query filter pattern (regexp) 
 +we use php preg-replace function to replace non-valid characters in user input 
 +ie when a surfer searches on your site 
 +a good pattern to start is 
 +/[^\p{L}\p{N}\s]/u 
  
 </code> </code>
 +===== Multiniche trade =====
  
-and then goes the rest of template with galleries +Let's look at what is in the script for maintaining multi-niche sites with a new rotation, except for the direct presence of categories.
  
-<code> +**Incoming redirect to a niche page** - the script can redirect traffic from a trader to the desired niche on your website ([[Tube Cms Pages#redirects]]).
-<thumb num=1-100>+
  
-<url> +**Sending a click to a trader’s niche page** - if a trader cannot automatically redirect your hits to the desired page, you can set up such a redirect yourself, although this is somewhat more complicated than simply enabling the option. First, you need to ensure that when you click, the script understands which group it is dealing with, so add &group=... to the output. There are 2 options: 
-    <loc>http://<?=$_SERVER['HTTP_HOST']?>/gallery/<!--GALLERY_SLUG-->/index.html</loc> +  - edit the template and make links like /gallery/asd/index.html?12x12x1234&group=GROUP NAME (substituting the corresponding tag in each template) 
-    <lastmod><!--ACTIVATION_DATE--></lastmod> +  - or enable the option ([[Tube Cms Pages#trade_by_groups]]) and determine the group by referrer. Then, in each trader’s settings, you can set Special Urls for each group. So if group A has the URL http://trader/a/ and the out parameters determine that the current group is A, the trader will be sent to http://trader/a/ instead of the generic http://trader/
-    <changefreq>daily</changefreq> +
-    <priority>0.5</priority> +
-</url>+
  
 +
 +**Note** what they forget when analyzing these functions
 +
 +**Trader 1 is in group A, but I see in the referrals of trader 1 referrals from my site from category B - he does not trade in groups!**
 +Usually the toplist is built globally, not by group, and then included on all pages of the site. That means page B will still show the top of all traders, even traders that should only appear in group A. [[Toplist]]
 +
 +In addition, if there are, for example, 2 traders in group A, the surfer has visited both, then when clicked it will be sent to all traders. If there are no active traders in the group and the surfer has not been there, he sends them to everyone.
 +
 +
 +
 +====== Performance ======
 +
 +Rotation requires a large number of pages.
 +
 +Here is the usual situation: you have a 200k gallery in 300 categories, let there be an average of 7 pages per category, 3 sorting options (ctr, data, duration) = 6300 possible pages + 200,000 gallery pages themselves.
 +
 +Generating that many pages as static files via cron would make little sense, so the pages are generated dynamically and cached instead. You control the cache time in the config. The shorter the cache time, the more pages the script must regenerate every second, so you directly affect server load.
 +
 +In addition, there is a so-called “warming up” of the site. For example, you just collected everything and are going to send traffic. If you pour in a lot of traffic at once, for example, immediately force 5k there - this 5k will spread across all pages and the script will need to quickly create all these pages, because you have just assembled the site and there is no cache for many pages. Therefore, when overclocking a new site, in order to avoid unnecessary load on the server, it is important to overclock it little by little. First, for example, you can drain 1-2k in an hour, during this time a cache will be created for the main pages, and then you can pour as much as necessary.
 +
 +
 +How much traffic the server can handle: after the cache is created, the pages are rendered conditionally instantly. It looks like this: 1 person came to the new site for the index, we created an index page and put it in cache for the time specified by you in the config. All other visitors will receive this page from the cache with virtually no load on the server. Even if 1M users come to this page during the cache time, they will receive the page without any problems.
 +
 +If we have just 1 website, let’s say 50k or 500k traffic - it doesn’t matter, because anyway we need to create almost the same number of pages for both 50k and 500k.
 +
 +But if you have 50 sites with 10k traffic each, in terms of load this is not the same as 1 site with 500k, because for 50 sites you need to create 50 times more pages than for one (with the same database size, number of categories, etc.).
 +
 +The size of the database also directly affects the load: if you have 100k in the database, then depending on the design it will be the same 3000 pages, 200k - 6000 pages.
 +
 +Translations and slave sites are technically similar to adding one more site, because a translated version or a slave with different descriptions or design still needs its own page generation. The difference from a completely separate second site is that some data is shared. So two fully different sites with 100k galleries each are not the same as a master with 100k galleries plus a slave, because in the master/slave case some information overlaps and the database can reuse part of it, which means less load.
 +
 +
 +It is impossible to answer the question “how much traffic a server with config X can handle” because it is not clear how many sites it is, what kind of design, etc., etc., can only be determined experimentally. Of course, we make every effort to optimize the speed of the script as much as possible.
 +
 +Here it is worth remembering that the server is overloaded like an avalanche. This means that it won’t happen that, for example, the server is running at 100% load and everything just starts to slow down a little. At some point, requests accumulate and the server crashes. The easiest way is to imagine how you rev ​​up the engine of a car, it works at 20%, then in the red zone, it works at full capacity, and then, if you add more, it’s not the power that drops due to overload, but the engine jams and it doesn’t work at all. Therefore, server monitoring is important.
 +====== Performance Log ======
 +
 +Templates often include a lot of their own PHP code, and at some point it becomes not entirely clear what exactly loads the server more: the script or its own code. To roughly look at this in the rotation settings there is
 +
 +Performance Log
 +Writes performance info into logs/perfomance.log
 +Adds some load at HDD as it writes each hit.
 +
 +When turned on, it writes to logs/perfomance.log, which creates a small additional load, so you shouldn’t leave it turned on unnecessarily. However, you can turn it on for a while to check the current state.
 +
 +The main advantage is that later in Maintenance -Logs you can look at this log and the script will automatically group the indicators so that you can see general statistics, and not just a lot of text. The names of the fields there change as this log is added, but in 99% of cases it is not a problem to understand what each entry means. If something is not clear, you can always ask on the forum.
 +
 +
 +
 +====== Custom gallery URL Refresh ======
 +
 +The classic URL looks something like http://domain/gallery/asd.html, which through rewrite turns out to be out.php?url=content&slug=asd, while skimming is taken from the default or can be set in rewrite, the main thing is that when refreshing the page, the gallery can be directed to the trader.
 +
 +It is impossible to avoid this automatically because a refresh is when the browser sends exactly the same request and there is no way to distinguish it on the script side.
 +
 +In general, I don’t consider this a problem because only the webmaster himself refreshes the gallery during the design process, but if you want to avoid this, you can add code to the page that will be executed when refresh is pressed
 +
 +https://stackoverflow.com/questions/18457797/how-to-know-whether-refresh-button-or-browser-back-button-is-clicked-in-firefox
 +
 +instead of alert("refresh button is clicked"); you can put your own code that will redirect for example to http://domain/gallery_PERMANENT/asd.html , and in rewrite add gallery_PERMANENT which will make a URL with &p=100
 +
 +====== Site Internationalisation (i18n) ======
 +
 +Translation of the site into other languages.
 +
 +
 +===== Site menu =====
 +
 +It’s quite easy to translate the site menu (those links like Most Popular, Order By date, etc.) into other languages. For example, we have the phrase “Most popular”. And several languages ​​for which this phrase needs to be translated.
 +
 +
 +Rotation - CMS - TPL Custom Vars create the most_popular variable, it can be used in the template as <!--CUSTOM_VAR_MOST_POPULAR--> (this is signed directly with the created variable). There you can also specify the translation for language slaves.
 +
 +
 +
 +
 +
 +====== Case Sensitive URLs ======
 +
 +According to the URL standard - case sensitive, whether the letter is large or small - matters. However, for some reason, most people think that it is not, so URLs get into the database in different versions, and questions arise: “why am I looking for http://domain/aaa and not finding it? When there is an entry in the database http://domain/AAA.. To avoid this, by default in the database the URL field is case insensitive so that there are no questions with the search.
 +
 +However, it happens that the URLs are really different and the database needs to distinguish the case. To do this, you need to run in the database
 +
 +  ALTER TABLE `rot_gallery_info` CHANGE `url` `url` VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL; 
 +  ALTER TABLE `rot_gallery_info` CHANGE `source_url` `source_url` VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL; 
 +  
 +and remember that after this the URLs from the example above will become different and when searching for URLs you will need to look at the case of letters.
 +
 +
 +====== How to move thumbs ======
 +
 +By default, thumbs are saved in /scj/thumbs. If you want to change the thumb storage location after adding content, then you need to
 +
 +- physically move thumb from scj/thumb to a new location 
 +  - change URL to thumbs in CJ settings - grabber settings
 +
 +====== Full page cache ======
 +
 +For highly loaded sites, page cache in its already completed form helps significantly. For example, we have a template
 +
 +<code>
 +
 +<thumb num=1-2>
 +<!--THUMB_URL--> 
 </thumb> </thumb>
 +
 +<?php
 +echo "date: " . date("Y-m-d");
 +?>
  
 </code> </code>
  
  
-In case if you want to output let's say categories just replace it with a proper tag+The script parses the template and replaces it with data from the database and stores the result in cache, so that the next surfer who looks at the same page will no longer have to replace the script tags
  
-For example+<code> 
 + 
 +/thumbs/img1.jpg 
 +/thumbs/img2.jpg 
 + 
 +<?php 
 +echo "date: " . date("Y-m-d"); 
 +?> 
 + 
 +</code> 
 + 
 + 
 +with the PHP code cache in its original form. For each surfer, this PHP code will be executed. 
 + 
 +In most cases, these are trifles in terms of load, but with a lot of traffic, when you need to reduce the load by a percentage, you can cache and PHP code, the page will be cached with the completed form, for example
  
 <code> <code>
-<category num=1-100> 
  
-<url> +/thumbs/img1.jpg 
-    <loc>http://<?=$_SERVER['HTTP_HOST']?>/category/<!--CATEGORY_NAME-->/</loc> +/thumbs/img2.jpg
-    <changefreq>daily</changefreq> +
-    <priority>1</priority> +
-</url>+
  
-</category>+date: 2023-01-01
  
 </code> </code>
  
-and so on. Don't forget to change urls and other parameters in subtemplate. 
  
-If you need to list category page you have to use <!--TOTAL_ITEMS--> tag to calculate how many page you have in each category+In most cases, this is ok, because the result of executing the code is the same for all surfers. 
 + 
 +However, there are rare situations when the template contains code that must be executed for each user, as it depends, for example, on the user’s current IP. There is an option for this 
 + 
 +  $config['full_page_cache'] = true;
  
-for example  
  
 +In this case, you can use the tag in the template
  
 <code> <code>
-<category num=1-100>+<php_code no_cache=true> 
 +<?php 
 +echo "user IP: " . $_SERVER['REMOTE_ADDR']; 
 +?> 
 +</php_code>    
 +</code>
  
-<url> +and it is this code that will not be cached.
-    <loc>http://<?=$_SERVER['HTTP_HOST']?>/category/<!--CATEGORY_NAME-->/</loc> +
-    <changefreq>daily</changefreq> +
-    <priority>1</priority> +
-</url>+
  
-then list pages 
  
-<? +For example template
-$per_page = 100;+
  
-for ($i = 1; $i <= ceil('<!--TOTAL_ITEMS-->'/$per_page); $i++ )  { +<code>
-?>+
  
-<a href='/category/<!--CATEGORY_NAME-->/<?=$i?>/'>Page <?=$i?></a>+<thumb num=1-2> 
 +<!--THUMB_URL-->  
 +</thumb>
  
-<? +<?php 
-}+echo "Rand: " . rand(0, 100);
 ?> ?>
  
 +<php_code no_cache=true>
 +<?php
 +echo "user IP: " . $_SERVER['REMOTE_ADDR'];
 +?>
 +</php_code>   
  
  
-</category>+</code> 
 + 
 +will be cached as 
 + 
 + 
 +<code> 
 + 
 +/thumbs/img1.jpg 
 +/thumbs/img2.jpg 
 + 
 +Rand: 22 (here you should pay attention that this PHP code is cached in its completed form) 
 + 
 +<?php 
 +echo "user IP: " . $_SERVER['REMOTE_ADDR'];    
 + 
 +// and here is the PHP code in its original form and will be executed for each surfer 
 + 
 +?> 
  
 </code> </code>
  
  
-===== Why order of thumbs differs in admin area and on site ===== 
  
-In most cases order should be the same order but you should keep in mind few settings:+Something to pay attention to when changing the template for full_page_cache. For example, your code is
  
-  - Test positions start (do not test new thumbs at positions higher then ) ie if a thumb is considered as new if won't be shown in the top position on site, but you can see it the top position in admin are +<code>
-  - % of test places on page (% of places on each page where we test NEW thumbs ) some thumbs might not be in the same order as well +
  
 +<?php
 +$img_src_hash = md5($_SERVER['REMOTE_ADDR']);
 +?>
  
-===== eval()'d code =====+<img_src='get_img.php?hash=<?=$img_src_hash?>'
  
-If you see something like "Parse error: some error, in /some/path/scj/tube/index.php(0) : eval()'d code on line 1717" - it means you have an error in php code in your template+some more HTML
  
-Check php tags like (<?php vs <?), commas. Or  you can remove your phpcode peace by peace to see what part of code causes the error.+</code>
  
-===== How to import a large cvs =====+We cannot cache it completely because this code must be different for each surfer. Whatever hash or cache you want to do
  
-Note, with version 2.X you can add as large cvs as you want. Written below relates to 1.X only.+<code>
  
-There's a logical issue with large cvss.  +<php_code no_cache=true> 
-A file can be too large to be uploaded with browser, it can take too much time to process it that causes timeouts and so on.+<?php 
 +$img_src_hash = md5($_SERVER['REMOTE_ADDR']); 
 +?> 
 +</php_code>   
  
-So some people decided to use importsets to inport large file and it would be file if they don't forget about it. 
  
-A common issue that 90% forget about those importsets and the script tries to import let's say 500 000 galleries every hour and that takes a lot of resources every hour.+<img_src='get_img.php?hash=<?=$img_src_hash?>'
  
-Besides that when you import a large file at once it creates a spike in server load that might cause a notable slow down in server's performance. +some more HTML
  
-So here's a good way to import large files:+</code> 
 + 
 +however, you need to remember that get_img.php?hash=<?=$img_src_hash?> will be stored in cache already executed, for example as get_img.php?hash=322jjJ!...., so any part that must output runtime data must also be treated as dynamic.
  
-  * create a file called cut.php  
  
 <code> <code>
  
 +<php_code no_cache=true>
 <?php <?php
 +$img_src_hash = md5($_SERVER['REMOTE_ADDR']);
 +?>
  
-error_reporting(E_ALL & ~E_NOTICE);+<img_src='get_img.php?hash=<?=$img_src_hash?>'
  
-if (!$_GET['file']) die("\n You have to pass file=... parameter in URL"); +</php_code>   
-if (!file_exists($_GET['file'])) die("\n File {$_GET['file']} does NOT exists. Check path. "); +
-if (!is_writeable($_GET['file'])) die("\n File {$_GET['file']} is NOT writeable. Set 666 permissions. ");+
  
-if (!file_exists('./tmp.file') or !is_writeable('./tmp.file')) die("\n File tmp.file does not exists OR is NOT writeable. Set 666 permissions. "); 
  
-echo passthru("head -n500 {$_GET['file']} "); +some more HTML
-if (!$_GET['test']) exec("tail -n +500 {$_GET['file']} > tmp.file; cp tmp.file {$_GET['file']} ");+
  
 </code> </code>
  
-  * upload a cvs file to the same location and chmod 666 it 
-  * Upload emtry file tmp.file and chmod 666 it as well 
-  * Add ImportSet with URL http://yourdomain/path_to/cut.php?file=cvs_file_name&test=true 
-  * Click Test and select fields 
-  * cut.php outputs  500 lines and cuts out thoses 500 files from cvs file. So on each run you'll add 500 new galleries.  Note, you can change this amount in the code above. 
-  * &test=true means "do not cut out those 500 lines". That's necessary for test purposes.later you should remove it and it should be like http://yourdomain/path_to/cut.php?file=cvs_file_name 
  
-That's it.+**Probable errors**
  
-So this was an entire large cvs file will be addded and even if you forget to remove that import set it won't create any load on a server.+You cannot include <php_code no_cache=true> in another block <php_code no_cache=true>. For example,
  
  
-===== Lazy load, load on scroll, pinterest and so on  =====+<code>
  
-The base idea here is to load some content when user scrolls down the end of a page. So user doesn't have to reload an entire page but gets "infinite" scroll instead. +<php_code no_cache=true> 
 +<?php 
 +$img_src_hash = md5($_SERVER['REMOTE_ADDR']); 
 +?>
  
-Let's sat we have a template "index" with 50 thumbs+some HTML
  
-<code> +<php_code no_cache=true>
-<thumb num=1-50> +
-<a href='/...'><img src='<!--THUMB_URL-->'></a> +
-</thumb> +
-</code>+
  
 +Another code
  
-In this example we use jquery.wookmark JS library (example here http://demo.smartcj.com/scroll/) Actually there are a plenty of libs like this one+</php_code>   
  
-When user reaches the end of a page we need to load some new content so here's an URL for it+more HTML
  
-  var P_BASE = '/?force_template=index_scroll_ajax&page='; 
-   
-   
-as you can see we load template "index_scroll_ajax" with a variable &page=.. So what jquery.wookmark does it changing that last parameter &page= each time loading /?force_template=index_scroll_ajax&page=2, /?force_template=index_scroll_ajax&page=3 and so on 
  
-Template index_scroll_ajax where we output 10 thumbs+</php_code>    
 + 
 + 
 +some more HTML
  
-<code> 
-<thumb num=1-10> 
-<a href='/...'><img src='<!--THUMB_URL-->'></a> 
-</thumb> 
 </code> </code>
  
  
 +In this case it's obvious, but there might be an option
  
-===== Thumbs\galleries are not being deleted ===== 
  
-Thumbs are not deleted right away when you click "delete" in admin area because we need to reload a page right away (to avoid timeout). So instead щи being deleted thumbs are marked as 'to delete'. Every 10 minutes rotation.php crontab job checks for thumbs with this mark and actually delete them. 
  
-if you want to force deletion:+<code>
  
-<code>cd /PATH_TO_/scj/bin/; env HTTP_HOST=yourdomain.com php rotation.php process_deleted=true </code>+<php_code no_cache=true> 
 +<?php 
 +$img_src_hash = md5($_SERVER['REMOTE_ADDR']); 
 +?>
  
 +</php_code>   
  
-===== It does not grab this gallery  ===== 
  
-Some galleries might have some peculiarities and are created in a way that the script can not download it.  We try to implement it as good as we can but if there's a gallery you'd like to add but the script can not parse it - you can use so called 'gateway' for it:+<!--INCLUDE_TEMPLATE... 
  
-  * let's say a gallery http://gallery/ contains an image http://gallery/image.jpg but the script can not see it +some more HTML
-  * we create file http://your_server/gate.php +
-  * and import the gallery  as http://your_server/gate.php?url=http://gallery/  +
-  * your script gate.php gets URL,  parses the real gallery and outputs it in a regular way , for example +
  
-<code> 
-<a href='http://image1'><img src='http://thumb1'></a> 
-<a href='http://image2'><img src='http://thumb2'></a> 
-and so on 
 </code> </code>
  
-**ImportSets** 
  
-Let's say we have importset http://sponsor/dump.php that outputs something like +in the template you include there is also <php_code no_cache=true>
  
-url|thumb_url|some_id 
  
-and we'd like to replace some_id some something more meaningful like http://test/modified_id+====== GET variables pass-through ======
  
 +For example, you received a hit with the parameter
  
-  * gate.php?url=http://sponsor/dump.php +  http://your_domain/?param=123 
-  * the gate gets URL, downloads it and replaces some_id with http://test/modified_id and outputs, so the script get's dump in a way you want it to be +   
-  * we add gate.php?url=http://sponsor/dump.php in import set instead of http://sponsor/dump.php+you want to output this param where on the page what would happen, for example 
 + 
 +  <img src='https://your_tracker/?param=123'> 
 + 
 +all you need to do is in the template 
 + 
 +    <img src='https://your_tracker/?param=<!--GET_param-->'> 
 +     
 +GET_param - will be replaced with the current value 
 + 
 +====== Reset Models  ====== 
 + 
 +If you already have a gallery database, let's say the gallery descriptions contain model names. The models themselves are simply added to Rotation - Models. 
 + 
 +The task is to assign gallery models (Rotation - special - Reset models) 
 + 
 +The search for a model name is done through the built-in search, the settings of which are located in Rotation - Settings. 
 + 
 +There are 3 options 
 + 
 +1. Just mysql - search by “model_name”. It is important to note 2 spaces here, because the model name for example Ola can be part of ebola for example. The problem is that in the default version mysql cannot search even in case sensitive 
 + 
 +2. Mysql fulltext - this search can already search using capital letters 
 + 
 +3. Sphinx can search case-sensitive and even different endings
  
 +Therefore, if you need better model assignment + better search in general, you must at least enable mysql fulltext
  
new_rotation_faq.1583903740.txt.gz · Last modified: by admin