====== New Rotation FAQ ======
===== What is mod_rewrite =====
mod_rewrite is a module for Apache that allows you to make "beautiful" URLs. The same module is available for both nginx and lighttpd.
For example, default template URLs look like http://domain/gallery/cool-gallery/index.html, even though no such physical path exists on the server. The rewrite rule converts that URL into something like http://domain/scj/tube/?slug=cool-gallery. The human-friendly URL is there for search engines and users.
The .htaccess file created during setup contains these rewrite rules.
A simple explanation of how it works:
As mentioned above, /gallery/cool-gallery/index.html does not really exist on the server. To open that gallery directly, the real URL would have to be something like /scj/cgi/out.php?url=content&slug=cool-gallery, which is not very clean. So we add a rewrite rule to .htaccess that tells Apache how to transform one URL into another. In this case the rule is:
RewriteRule ^gallery/(.*)/index.html$ /scj/cgi/out.php?url=content&slug=$1
This means: if the URL starts with gallery/, then has some value captured by (.*), and ends with /index.html, it matches this rule. Each captured part gets its own number: the first one is $1, the second would be $2, and so on.
In this example, $1 becomes cool-gallery. Apache therefore rewrites the URL to /scj/cgi/out.php?url=content&slug=cool-gallery and the gallery is shown.
URLs like /gallery/cool-gallery/index.html appear on pages because subtemplates contain markup like this:
href="/gallery//index.html
So if you want to change the URL format on the site, you need to change both the subtemplate and the rewrite rule so they match each other.
===== Why in Rotation - Groups does it show 10 gallery in my group, but on the site page or Rotation - List thumbs 20 =====
One gallery can belong to more than one group (category). You can see this when editing any thumb: it has one main group and multiple extra groups. Rotation - Groups counts only by the main group, so you can clearly see how many unique galleries there actually are.
For example, we have 2 groups A and B. and 2 galleries - one has main group A, the other has main group B. But both have A and B marked in the ext groups field.
Rotation - Stats will show that we specifically have 2 galleries, one in group A and one in group B.
On the site, in each of the groups you will see 2 galleries.
===== Custom gallery with my local content =====
- Upload anywhere thumb
- when importing, specify Image List = listing of URLs of images
===== What is Init cats in Rotation - Special or why I added a category in the admin panel, but it didn’t appear on the site =====
Recalculating categories, namely choosing the best category thumb and counting how many thumbs there are in a category, is a rather difficult task for the server, so it does it once every 15 minutes, stores the result in cache and all other parts of the script use cache. Init cats is a forced launch of this operation.
If you have added a new category, it will appear in a maximum of 15 minutes, if you need it faster, click this button.
===== What is cache =====
The script does not generate static HTML pages (see the question about mod_rewrite above), all pages are dynamic. In order not to recreate the page for each user, the script creates it once and saves it in cache for a certain time (CACHE_TIME), the next visitor during the lifetime of the cache will receive this page from the cache, without loading the server.
The cache time is set in common.php CACHE_TIME, starting from update 46, the time in common.php can not be specified, but can be specified in the rotation settings. Please note that the time specified in common.php takes precedence; if specified in common.php, the script ignores the rotation settings.
In order to view a page without cache, you need to add the skip_cache=true parameter to the URL. For example, http://domain.com/?skip_cache=true
Starting with update 46, recreate cookie appeared - Rotation - Special - Recreate visited pages - the script places a secret cookie in your browser for 10 hours (you can remove the cookie at any time there). If you access any page of your site with this cookie, this page will be recreated (the cache for this particular page will be reset). It can be useful when changing the design of one specific page.
The difference between skip_cache and recreate cookie is that skip_cache will show the page without cache only to you and that’s it, and recreate cookie will show the page without cache, then it will be put in the cache as you see it and other users will see it from the cache in this form.
Once again: the page was created at 00 minutes. Cache time 10 minutes. in 05 you change the design.
if you looked at it in 05 with skip_cache, you will see a new design, but a user who came in 06 (before the cache expires) will see the old design.
if you open a page with a recreate cookie in 05, then the user who came in 06 will see the new design.
===== Cache Engines (New) =====
The basic principle of the engine is to generate the desired page, put it in the cache for a certain time and display it until the cache is cleared. What engines are currently available:
**File**
Basically cache is stored in files (scj/cache - file cache).
Pros - it works everywhere and at once.
The disadvantages are that it is a file cache, it is not as fast, the cache system is not as efficient as we would like, and you have to separately take care of the size of the cache folder.
**Memcached**
At one time, this was certainly a big step forward in terms of cache.
Pros - everyone knows it, admins install it on hosting without any problems, it’s easy to administer, etc.
Disadvantages - it is almost impossible to get any understanding of the actual memory occupied; when the server is overloaded, the entire cache is deleted and you need to “warm up the cache” to put the most used data there. When the server is overloaded due to load, this is a serious problem - the server is already loaded, and then all the cache has disappeared.
All data is cached equally - it doesn’t matter if it is the most used or used once an hour - they all hang in the same memory and the service does not know how to dump them to disk, for example.
With memcache, you cannot clear cache for one site only. All sites share the same cache space, so clearing it affects every site at once. You can of course run multiple memcache instances on different ports, but that usually requires admin help and is inconvenient in practice.
How to install:
- ask the admin to install Memcached
- register memcache in the config
$config['memcached_host'] = '127.0.0.1';
$config['memcached_port'] = '11211';
**Therefore, now there are new engines that are recommended for use**
In new engines, which are basically called NoSQL solutions, there are many more possibilities than will be described below, but for current cache purposes the basic ones will be described.
The easiest solution to switch to is **Couchbase**
**CouchBase** is a continuation of MemBase, apparently the closest descendant of memcachedb, which in turn comes from memcached.
The main thing is that practically no movements are required from you in order to start using it. You install CouchBase and it works exactly the same as memcache. In order to start using it you need
- ask the admin to install CouchBase
- add extension couchbase to PHP
- register in config.php
$config['couchbase_host'] = '127.0.0.1';
$config['couchbase_port'] = '11210';
$config['couchbase_username'] = 'testuser';
$config['couchbase_password'] = 'testuser';
$config['couchbase_bucket_name'] = 'test';
and that's it, you are already using a modern version of memcache. You can use this even with version 48-49.
**How is this better than memcached**
* There is no concept of "warming up cache". In fact, this is a file cache, but made an order of magnitude more correctly, which loads the disk much less, calculates when and how best to write data in blocks to disk, etc. In this case, the data lies in several files, and not in hundreds of files like a file cache.
* Parsed data disappears from the disk/memory on its own; you don’t have to worry about it yourself.
* You can, just like with memcache, limit the amount of memory allocated for cache, but cache will not be limited by this amount. Even if there is more data, it will leave the most active ones in memory and download the rest to disk. And loading the required element from disk is faster than generating it again.
* You can quickly and easily scale the system from the web admin panel.
* You can also create several separate caches. In Couchbase this is like running several memcache instances, but everything is managed from the web admin panel, which is much faster and more convenient. For example, you can create 10 caches, each on its own port, and assign a separate port to each site. In that setup, the long-requested “Delete all cache” button effectively becomes possible, because you can clear the full cache for one site only.
* When moving, cache is easy to copy because it is not a lot of small files that are copied, as with a file cache, but one database.
**Redis** is a separate NoSQL database project written from scratch. Its meaning for our purposes is almost the same (in reality, couchBase is more of a document oriented storage, and Redis is a purely key-value storage). To work with it, functionality was added in version 50.
In order to start using it you need
- ask the admin to install Redis
- ask him to install the redis module for PHP
- register in config.php
$config['redis_host'] = '127.0.0.1';
$config['redis_port'] = '6379';
$config['redis_database'] = 0;
$config['redis_password'] = '';
if redis is hanging on a socket it will look something like this
$config['redis_host'] = '/tmp/redis.sock';
$config['redis_port'] = '0';
and that's all.
In this config, in addition to the clear fields, there are several new ones. redis_password - in most cases it is not relevant since now everything is on dedicated servers.
redis_database - Redis does not have alphabetic database names, but rather numbers. If you have several sites, you can give each site its own number. But you don’t have to specify it and use one database for all sites. The difference will be that if you want to reset the entire cache and you use one database for all sites, the cache will be deleted for all sites. If each site has its own database, the cache will be chipped in for one.
**How is this better than memcached**
* All the same advantages apply to couchbase.
* Redis is a more cache-specific solution, but CouchBase has a ready-made convenient admin panel (talking about version 2+).
===== Where is cache stored? (memcache) =====
By default, a file cache is used (stored in /scj/cache), but you can also use, for example, Memcache. To do this, you just need to add it to scj/includes/config.php
$config['memcached_host'] = 'localhost';
$config['memcached_port'] = '11211';
The address and port may change (check with your admin), but in most cases they will be the same.
It makes sense to configure memcache to work via socket - it's faster. In this case, the config will be something like this
$config["memcached_host"] = "unix:///tmp/memcache.sock";
$config["memcached_port"] = 0;
There are several nuances:
- file cache is not as slow as it may seem. In practice, the difference is often smaller than people expect.
- with file cache, the cache stays on disk. If the server is overloaded or restarted, the cache does not simply disappear and the script does not have to rebuild everything from scratch. With memcache, the entire cache can be lost, and then the script must regenerate many pages on the fly, which creates extra load again. If you do use memcache, make sure it has enough memory. The exact amount depends on database size and on how heavy your templates are, since the cache stores the generated HTML itself. As soon as memory runs out, pages start being regenerated on the fly. Memcache size also matters because old entries are evicted when space is needed. For example, if the cache can hold 10 one-megabyte entries, adding an eleventh entry pushes the oldest one out. Another issue is fragmentation: memcache stores data in blocks, so real memory usage is higher than the reported payload size. Roughly speaking, if a block is 100 bytes and you save 1 byte, statistics may report 1 byte while 100 bytes are actually occupied. If an entry is evicted, memcache does not report an error back to the script, so the script cannot react to that directly. A rough practical estimate is about 15% fragmentation.
===== Why in my admin panel and on such and such a page of the site the thumbs are in a different order =====
With the same sorting, the order should be the same, taking into account the following nuances:
1. in the admin panel it shows the name as it is in the database according to CTR, for example, on the site page in some places there may be “new thumbs” that are being tested. For example, on the page thumb 123 is in 15th place, and in the admin panel it is in 50th place. This is probably a new thumb that the script is testing.
2. The new thumb may get an unexpectedly high CTR. For example, they showed it 2 times and clicked on it 1 time = CTR 0.5, but 2 impressions are not a reason to judge that the thumb is good. Therefore, by default, “new” thumbs are not shown in first place. You can change this in settings (New Rotation: New thumbs placement : Place new thumb (shows less then New thumbs timelive) on places other then Test positions start if it has good CTR. ). It’s easy to check if you select New Thumbs = dont show new in List Thumbs.
===== Why is there an empty tag cloud on my page =====
A tag cloud is generated from active tags every 30 minutes and stored in cache. Accordingly the options:
* no active tags
* less than 30 minutes have passed
* you nailed cache
===== Thumbs (galleries) are not deleted =====
* previously there was a problem when deleting a large number of thumbs. For example, if one sponsor had 10k thumbs and you deleted that sponsor, the script tried to remove everything immediately and could hit a timeout. The issue was even worse if you needed to remove not just thumbs, but a full custom gallery.
* now thumbs are deleted by cron. When you click “delete” in the admin panel, the thumb is not deleted instantly. It is moved to the 'Marked for deletion' status, and every 10 minutes cron checks for such items and removes them
* if you need to delete “right now” - run rotation.php with the parameter process_deleted=true
* cd /PATH_TO_/scj/bin/; env HTTP_HOST=yourdomain.com php rotation.php process_deleted=true
* the only exception: if you delete a thumb in Rotation - List thumbs and less than 30 thumbs are deleted, then the deletion occurs immediately.
The deletion speed is hard to predict and depends on the number of galleries involved, current server load, whether a custom gallery is involved, and whether the files are on the current server or on FTP.
===== How much traffic is needed to rotate thumbs =====
It is not necessary to revise everything to the end for good sales, but in general it is very easy to calculate how much traffic you need.
A thumb is considered rotated after it has been shown New thumbs timelive times, which is 500 by default.
Let’s say there are 200 thumbs on the page, and the default percentage of test positions on the page is 15. That means new thumbs will be shown in 30 of those 200 positions.
Every visit to the page means +30 impressions for new thumbs.
For example, we have 10,000 thumbs, to rotate them we need 10,000*500 = 5,000,000 impressions. One visit to the page is 30 impressions, but to rotate the base to 10k we need 5M / 30 = 167k page loads.
===== Not every thumb of a gallery is tested =====
Apart from the banal lack of traffic, it usually means that the gallery has, for example, 10 thumbs, but only one has impressions. How this happens: for example, we have 100 galleries of 10 thumbs, 20 thumbs on the page. Let's start rotation. All have a CTR of 0. We displayed the first thumb with gallery ID 1 - 20. They received a CTR greater than 0. We need to refresh the page. test ones for example 5 on the page.
We display IDs 1 - 15 as the 15 best (and so far only the first thumb from the gallery has received CTR and impressions). And then 5 test ones - here we can take thumb number 2 with gallery ID 16-20
It turned out that these thumbs scored less than thumb 1 gallery ID 1-15, we take for example the 3rd thumb with gallery ID 16-20.
Thus, it turns out that gallery ID 1-15 has impressions only for the first thumb. When their CTR is less than the CTR of others, they will be “squeezed out” from the top.
They also have the opportunity to be shown on page 2 of the pagination, but again, provided that your traffic reaches the 2nd page.
===== Parameter out.php &link= ... =====
Initially, out.php has a &link=blabla parameter, the meaning of which is that statistics on blabla can be seen Stats - Links. But rotation uses it to pass rotation parameters (12x123x1234). This looks very simple in practice: for example, the link /gallery/cool-desc/index.html?12x345x567 rewrite (see what it is above) is converted into a URL like out.php?slug=cool-desc&link=12x345x567 and later the script processes statistics on links, converting this into statistics on thumb. Accordingly, if you have a URL like out.php?member=domain.com&link=top, then this option still works the same, but it cannot be used for rotation links. In order to solve this situation, another parameter &link2 was introduced, the meaning of which is that you can do out.php?slug=cool-desc&link=12x345x567&link2=blabla and still view statistics on links and rotate the thumb at the same time.
===== What are thumb categories =====
A category thumb is the default thumb for a category. However, you can configure the script to use a thumb that is not the absolute best one, for example the 2nd or 3rd by CTR. These settings are available in Rotation - CMS - Settings.
===== How to create a master site from a slave =====
- Dump the master database
- We look in rot_linked_db for the number of the slave that we are disconnecting (X)
- Disconnect the desired slave from the master
- Fill the database with the former on the left (dump) from point 1, but only the tables rot_gallery_info, rot_gallery_statsX, rot_gallery_dataX and rot_thumbs
- rename rot_gallery_statsX to rot_gallery_stats1, rot_gallery_dataX to rot_gallery_data1
===== Gallery statuses =====
Pool - Active - Old - actually necessary for Shift settings to work in rotation groups. The point is to fill the site automatically. So you added a lot of galleries to the pool status, set 10 galleries to be added once a day and the site adds them from the pool to active ones, and they appear on the site. This gives the appearance of daily updates. Old - if you want to remove old galleries from the site while keeping the total number the same.
On the site, the listing shows only galleries with the active status.
preload - galleries are added to Preload (in the admin panel) so that you can choose which of the grabbed thumbs will eventually be on the site. Those that you do not select will be deleted.
to_grab - when galleries are added to the database they receive this status before the grabber creates a thumb for them. At the moment thumbs are created, they receive the processing status. to regrab - gallery is waiting for regrab.
to delete - gallery is waiting to be deleted. Many galleries are marked for deletion, and it is impossible to delete them instantly, especially
grab error - if something went wrong while grabbing thumb, the gallery is either deleted or given the grab error status depending on your settings.
Banned - the URL remains in the database and will not be added again. Like grab error, this only makes sense if you are adding a gallery via import sets and there is a possibility that some galleries will be added again and again. For example, you add a set through import, gallery goes to preload. You selected part of the gallery, but decided not to add some. An hour later, this import set is added again, and there is the same gallery again - it will again end up in preload and you will have to sift it out again. To avoid that, you can add it to banned. The downside is that the database in this case grows and there is a gallery that is not used, but takes up space in the database.
Inactive (the equivalent of hidden in version 1.X) means the gallery is not shown in listings, but is still available by direct URL. For example, imagine a gallery was indexed but hotlinked its content from somewhere else. Later the source content was deleted and the gallery stopped working. Previously, the checker could delete such galleries, or mark them inactive. The downside was that for CE spiders the old URL would now return 404. In some cases it is better to return a page without content instead of a 404. That is what the “Hidden” status is for: the gallery remains accessible by direct URL, but its thumb is removed from rotation. This status can be assigned by an import set that removes galleries.
Gallery Checker has a similar option.
The exact effect of this on the CE is unknown.
===== Category statuses =====
Categories have 3 statuses:
- active - displayed in , while the gallery may have more than one category and it is not clear which one should be displayed. Therefore, either a tag of the form or the construction ... is used
===== Group Deactivation (Group Exclusion) =====
Let's say we have a master on which there are groups A and B. We connect a slave on which we deactivate group B. When sampling for a slave, we have 2 options:
* selection of all gallery EXCEPT group B
* or all gallery from group A
at first glance they are the same thing. However, if our gallery is in both groups, then in one case it will be included in the selection, but in the other it will not.
So in settings there is a Group Exclusion setting
- Hard = the selection goes like “all the gallery EXCEPT group B” (those galleries that are in both groups are not included in the selection)
- Soft = the selection follows the second option, namely "all galleries from group A" (those galleries fall into the selection)
When is this needed? Imagine you have 3 groups: straight, guy, and teen. Suppose that on a slave site you want only straight content. In that case, a surfer interested in guy content should not see those galleries at all, so those groups become a strict exclusion.
===== It does not grab this gallery =====
Some pages may be designed in such a way that they cannot be raked automatically.
In this case, you can make a tn gate to transform it into a more understandable form:
* for example, URL gallery http://gallery/, thumb http://gallery/image.jpg
* we make file http://your_server/gate.php
* import dump from gallery to http://gallery/|http://your_server/gate.php?url=http://gallery/ (pattern url|thumb_url)
* your gate.php script receives a URL parameter and can pull out a picture from there that the smart will perceive as a regular picture and add to the database
**Example:**
Suppose we have a gallery at http://gallery/ with unusual markup:
The standard script parser cannot parse this, so we make the gate http://my_server/gate.php which parses this gallery and shows thumb from there (the code is simplified for the example, of course more checks are needed)
|", $html, $out);
$image_num = (isset($_GET['num'])) ? $_GET['num'] : 1;
if ($out[1][$image_num-1]) {
header("Content-type: image/jpeg");
echo file_get_contents($out[1][$image_num-1]);
}
}
check that the gate is working
http://my_server/gate.php?url=http://gallery/
http://my_server/gate.php?url=http://gallery/&num=2
At these URLs, the browser should display http://gallery/1.jpg and http://gallery/2.jpg from our gallery, respectively.
Now import this gallery, pattern: url|thumb_url
http://gallery/|http://my_server/gate.php?url=http://gallery/,http://my_server/gate.php?url=http://gallery/&num=2
As you can see here, the URL will be http://gallery/, and for thumb we will take the images that are visible at the URL http://my_server/gate.php?url=http://gallery/ and http://my_server/gate.php?url=http://gallery/&num=2.
Since we don’t know exactly how many pictures there will be in the gallery, we can add, for example, http://my_server/gate.php?url=http://gallery/&num=123 (that number is non-existent) to the thumb list - the script will simply not be able to grab it and will skip it.
**Option 2: replacement at the import stage**
Let's say we have import set http://sponsor/dump.php which produces a dump of the form
url|thumb_url|desc
and we decided to make a custom gallery from this content, but the problem again is that the gallery has the same appearance as in the previous example, but the grabber cannot grab it in normal mode.
Making a gate - option 2:
* gate.php?url=http://sponsor/dump.php
* the gate receives the feed URL, grabs it, processes each line as a gallery and adds a field with pictures for a custom gallery, which in this case the gate produces a dump immediately in the form of url|thumb_url|desc|**image_list**
* we add the set to the import immediately gate.php?url=http://sponsor/dump.php
**Option 3: Import using XPath**
For example, we have a new tube that gives a URL like http://tube/gallery.html,; it doesn’t have an export; we grab flv_url and flv_thumb_url from there and make our own custom gallery from them.
* If you don’t know what xpath is, you should read it, self-education is useful
* open your favorite browser - developer tools
* open the page, find the desired element and do Copy - XPath
* we get something like /html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2] , by now, if you read the first paragraph, you already understand that this is the path to the page element. This way the script will know where the element you need is located on the page. Of course, you don’t need the element itself, but its value, for example /html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href if it’s a link or @src if it’s an image, etc.
* add url|flv_url|flt_thumb_url as a pattern, marking those fields in which you need to extract Xpath as XPath:/.... in our version it turns out something like
http://tube/gallery.html|XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href|XPath://html/head/meta[5]/@content
If you insert this line into the import, the Test Xpath button will appear. If you click on it, the script will pull out the gallery URL and calculate the specified fields. This is necessary to check that you have entered xpath correctly.
This way you can add as many URLs as you like
http://tubesite.com/gallery.html|XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href|XPath://html/head/meta[5]/@content
http://tubesite.com/gallery222.html|XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href|XPath://html/head/meta[5]/@content
http://tubesite.com/gallery333.html|XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href|XPath://html/head/meta[5]/@content
and thus the gallery will be added. But it's certainly not very pretty. Therefore, you can add Import Replacements where
url contains = tubesite.com
replace FLV Url = XPath://html/body/div[3]/section/div[1]/div/div[1]/div[2]/a[2]/@href
This way, you can simply import http://tubesite.com/gallery.html and the script will already replace the necessary fields for this site during import.
As usual, the best approach is to add one gallery first and check the grab log.
**Option 4: ahhhh, everything is complicated, I don’t want to read and understand anything**
If you don't want to do it yourself, you can always pay and have the work done for you. Any schoolchild can make a gate or copy xpath in a browser. If you don’t want to contact schoolchildren, contact the support with specific sites. Estimated price - $100 per site.
===== Lazy load, load on scroll, pinterest and so on =====
All these options are based on the fact that when scrolling to the end of the page, new content is loaded and the user does not have the page overloaded, but new content is added at the end of the page.
It looks like this: imagine an index template with 50 thumbs
We add, for example, jquery.wookmark to the template (not the only option, of course) as in http://demo.smartcj.com/scroll/
upon reaching the end of the page it must load additional content, for this we indicate from where
var P_BASE = '/?force_template=index_scroll_ajax&page=';
As you can see, the index_scroll_ajax template is loaded and &page= is specified. Each time when the end of the page is reached, jquery.wookmark will load this URL changing the page number
Template index_scroll_ajax where we display 10 thumbs
===== Duplicate test thumbs =====
The problem manifests itself on a database where thumb has not yet gained CTR\impressions at all: we open page 1, then page 2 (of the same sample) and we see repeated thumbs. The following happens:
For example, imagine we have 3 thumb positions per page, and only 9 thumbs total. That means there are 3 pages.
test position -1 per page, 1 test 2 rotated thumbs.
On page 1: we take 2 thumbs by CTR, since all CTR 0 get thumbs 1 and 2.
We take the first test one - 3. It turns out 1 2 3.
Page 2 - we take the next 3 thumbs by CTR using the page offset, but since CTR values are still unstable, the sequence may become 4 5. Then we also need one test thumb, so the script again looks from the start, sees that thumb 1 still has no impressions and is still a valid test thumb, and produces 4 5 1. That is where the repetition comes from.
As soon as the database gains a little CTR, it turns out that according to the CTR, a thumb is enough to type a page and thus the thumbs are not repeated.
===== Fast Navigation =====
If you have a large database, then it makes sense to replace with just
The difference is that if we have navigation, we first need to calculate how many gallery there are for a specific request, this is a certain load on the database.
If we only have links to the next/previous pages, then there is no need to count anything in the database.
Please note that in this case the total_items tag will simply show the number of galleries on the current page.
===== User Thumbs =====
Suppose your import contains many thumbs, but there is not enough traffic to rotate all of them, which is the usual case. Still, you may want to keep all of them for rolling thumbs. That is what the User Thumbs option is for: the specified thumbs are downloaded, the crop profile is applied, but they are not added to the database. They are stored only on disk in /scj/thumbs/user_thumbs/.
Those we can import as URL|THUMB_URL|USER_THUMBS
If it is not possible to provide two separate thumb fields in the import, you can simply import as URL|USER_THUMBS and set How Many Thumbs from each gallery ?= X. In that case, all thumbs from this field are saved to disk, while X thumbs are added to the database for rotation.
This way the database does not grow and there is no need to rotate extra thumbs.
At the same time, you need to understand that the script does not know anything about thumbs, which are stored in user_thumbs, so for rolling thumbs you have 2 options. Either for each gallery the number of thumbs should be the same, or when rolling, the script should preload thumb for rolling and thus understand how many there are in total.
In the template, the URL to the directory with the user thumb for this gallery can be obtained with the tag
===== Filter request parameters =====
For example, you have a website in English and there is a search form where people enter search queries on the site. A log of search queries is displayed somewhere on the site. Some people try to search, for example, in Russian and this is displayed in the query logs, but you would not want this to be visible. You can filter unwanted characters at the request stage.
Add to common.php
if (isset($_GET['search'])) $_GET['search'] = preg_replace('|[^a-z,0-9,\s]|is','',$_GET['search']);
and these will only be Latin characters
There is also an option "Log empty search queries" and if nothing is found according to the request, it will not be logged.
After the update, an option was added to the rotation settings that does the same thing
Search query filter pattern (regexp)
we use php preg-replace function to replace non-valid characters in user input
ie when a surfer searches on your site
a good pattern to start is
/[^\p{L}\p{N}\s]/u
===== Multiniche trade =====
Let's look at what is in the script for maintaining multi-niche sites with a new rotation, except for the direct presence of categories.
**Incoming redirect to a niche page** - the script can redirect traffic from a trader to the desired niche on your website ([[Tube Cms Pages#redirects]]).
**Sending a click to a trader’s niche page** - if a trader cannot automatically redirect your hits to the desired page, you can set up such a redirect yourself, although this is somewhat more complicated than simply enabling the option. First, you need to ensure that when you click, the script understands which group it is dealing with, so add &group=... to the output. There are 2 options:
- edit the template and make links like /gallery/asd/index.html?12x12x1234&group=GROUP NAME (substituting the corresponding tag in each template)
- or enable the option ([[Tube Cms Pages#trade_by_groups]]) and determine the group by referrer. Then, in each trader’s settings, you can set Special Urls for each group. So if group A has the URL http://trader/a/ and the out parameters determine that the current group is A, the trader will be sent to http://trader/a/ instead of the generic http://trader/
**Note** what they forget when analyzing these functions
**Trader 1 is in group A, but I see in the referrals of trader 1 referrals from my site from category B - he does not trade in groups!**
Usually the toplist is built globally, not by group, and then included on all pages of the site. That means page B will still show the top of all traders, even traders that should only appear in group A. [[Toplist]]
In addition, if there are, for example, 2 traders in group A, the surfer has visited both, then when clicked it will be sent to all traders. If there are no active traders in the group and the surfer has not been there, he sends them to everyone.
====== Performance ======
Rotation requires a large number of pages.
Here is the usual situation: you have a 200k gallery in 300 categories, let there be an average of 7 pages per category, 3 sorting options (ctr, data, duration) = 6300 possible pages + 200,000 gallery pages themselves.
Generating that many pages as static files via cron would make little sense, so the pages are generated dynamically and cached instead. You control the cache time in the config. The shorter the cache time, the more pages the script must regenerate every second, so you directly affect server load.
In addition, there is a so-called “warming up” of the site. For example, you just collected everything and are going to send traffic. If you pour in a lot of traffic at once, for example, immediately force 5k there - this 5k will spread across all pages and the script will need to quickly create all these pages, because you have just assembled the site and there is no cache for many pages. Therefore, when overclocking a new site, in order to avoid unnecessary load on the server, it is important to overclock it little by little. First, for example, you can drain 1-2k in an hour, during this time a cache will be created for the main pages, and then you can pour as much as necessary.
How much traffic the server can handle: after the cache is created, the pages are rendered conditionally instantly. It looks like this: 1 person came to the new site for the index, we created an index page and put it in cache for the time specified by you in the config. All other visitors will receive this page from the cache with virtually no load on the server. Even if 1M users come to this page during the cache time, they will receive the page without any problems.
If we have just 1 website, let’s say 50k or 500k traffic - it doesn’t matter, because anyway we need to create almost the same number of pages for both 50k and 500k.
But if you have 50 sites with 10k traffic each, in terms of load this is not the same as 1 site with 500k, because for 50 sites you need to create 50 times more pages than for one (with the same database size, number of categories, etc.).
The size of the database also directly affects the load: if you have 100k in the database, then depending on the design it will be the same 3000 pages, 200k - 6000 pages.
Translations and slave sites are technically similar to adding one more site, because a translated version or a slave with different descriptions or design still needs its own page generation. The difference from a completely separate second site is that some data is shared. So two fully different sites with 100k galleries each are not the same as a master with 100k galleries plus a slave, because in the master/slave case some information overlaps and the database can reuse part of it, which means less load.
It is impossible to answer the question “how much traffic a server with config X can handle” because it is not clear how many sites it is, what kind of design, etc., etc., can only be determined experimentally. Of course, we make every effort to optimize the speed of the script as much as possible.
Here it is worth remembering that the server is overloaded like an avalanche. This means that it won’t happen that, for example, the server is running at 100% load and everything just starts to slow down a little. At some point, requests accumulate and the server crashes. The easiest way is to imagine how you rev up the engine of a car, it works at 20%, then in the red zone, it works at full capacity, and then, if you add more, it’s not the power that drops due to overload, but the engine jams and it doesn’t work at all. Therefore, server monitoring is important.
====== Performance Log ======
Templates often include a lot of their own PHP code, and at some point it becomes not entirely clear what exactly loads the server more: the script or its own code. To roughly look at this in the rotation settings there is
Performance Log
Writes performance info into logs/perfomance.log
Adds some load at HDD as it writes each hit.
When turned on, it writes to logs/perfomance.log, which creates a small additional load, so you shouldn’t leave it turned on unnecessarily. However, you can turn it on for a while to check the current state.
The main advantage is that later in Maintenance -Logs you can look at this log and the script will automatically group the indicators so that you can see general statistics, and not just a lot of text. The names of the fields there change as this log is added, but in 99% of cases it is not a problem to understand what each entry means. If something is not clear, you can always ask on the forum.
====== Custom gallery URL Refresh ======
The classic URL looks something like http://domain/gallery/asd.html, which through rewrite turns out to be out.php?url=content&slug=asd, while skimming is taken from the default or can be set in rewrite, the main thing is that when refreshing the page, the gallery can be directed to the trader.
It is impossible to avoid this automatically because a refresh is when the browser sends exactly the same request and there is no way to distinguish it on the script side.
In general, I don’t consider this a problem because only the webmaster himself refreshes the gallery during the design process, but if you want to avoid this, you can add code to the page that will be executed when refresh is pressed
https://stackoverflow.com/questions/18457797/how-to-know-whether-refresh-button-or-browser-back-button-is-clicked-in-firefox
instead of alert("refresh button is clicked"); you can put your own code that will redirect for example to http://domain/gallery_PERMANENT/asd.html , and in rewrite add gallery_PERMANENT which will make a URL with &p=100
====== Site Internationalisation (i18n) ======
Translation of the site into other languages.
===== Site menu =====
It’s quite easy to translate the site menu (those links like Most Popular, Order By date, etc.) into other languages. For example, we have the phrase “Most popular”. And several languages for which this phrase needs to be translated.
Rotation - CMS - TPL Custom Vars create the most_popular variable, it can be used in the template as (this is signed directly with the created variable). There you can also specify the translation for language slaves.
====== Case Sensitive URLs ======
According to the URL standard - case sensitive, whether the letter is large or small - matters. However, for some reason, most people think that it is not, so URLs get into the database in different versions, and questions arise: “why am I looking for http://domain/aaa and not finding it? When there is an entry in the database http://domain/AAA.. To avoid this, by default in the database the URL field is case insensitive so that there are no questions with the search.
However, it happens that the URLs are really different and the database needs to distinguish the case. To do this, you need to run in the database
ALTER TABLE `rot_gallery_info` CHANGE `url` `url` VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL;
ALTER TABLE `rot_gallery_info` CHANGE `source_url` `source_url` VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL;
and remember that after this the URLs from the example above will become different and when searching for URLs you will need to look at the case of letters.
====== How to move thumbs ======
By default, thumbs are saved in /scj/thumbs. If you want to change the thumb storage location after adding content, then you need to
- physically move thumb from scj/thumb to a new location
- change URL to thumbs in CJ settings - grabber settings
====== Full page cache ======
For highly loaded sites, page cache in its already completed form helps significantly. For example, we have a template
The script parses the template and replaces it with data from the database and stores the result in cache, so that the next surfer who looks at the same page will no longer have to replace the script tags
/thumbs/img1.jpg
/thumbs/img2.jpg
with the PHP code cache in its original form. For each surfer, this PHP code will be executed.
In most cases, these are trifles in terms of load, but with a lot of traffic, when you need to reduce the load by a percentage, you can cache and PHP code, the page will be cached with the completed form, for example
/thumbs/img1.jpg
/thumbs/img2.jpg
date: 2023-01-01
In most cases, this is ok, because the result of executing the code is the same for all surfers.
However, there are rare situations when the template contains code that must be executed for each user, as it depends, for example, on the user’s current IP. There is an option for this
$config['full_page_cache'] = true;
In this case, you can use the tag in the template
and it is this code that will not be cached.
For example template
will be cached as
/thumbs/img1.jpg
/thumbs/img2.jpg
Rand: 22 (here you should pay attention that this PHP code is cached in its completed form)
Something to pay attention to when changing the template for full_page_cache. For example, your code is
'
some more HTML
We cannot cache it completely because this code must be different for each surfer. Whatever hash or cache you want to do
'
some more HTML
however, you need to remember that get_img.php?hash= will be stored in cache already executed, for example as get_img.php?hash=322jjJ!...., so any part that must output runtime data must also be treated as dynamic.
'
some more HTML
**Probable errors**
You cannot include in another block . For example,
some HTML
Another code
more HTML
some more HTML
In this case it's obvious, but there might be an option
'>
GET_param - will be replaced with the current value
====== Reset Models ======
If you already have a gallery database, let's say the gallery descriptions contain model names. The models themselves are simply added to Rotation - Models.
The task is to assign gallery models (Rotation - special - Reset models)
The search for a model name is done through the built-in search, the settings of which are located in Rotation - Settings.
There are 3 options
1. Just mysql - search by “model_name”. It is important to note 2 spaces here, because the model name for example Ola can be part of ebola for example. The problem is that in the default version mysql cannot search even in case sensitive
2. Mysql fulltext - this search can already search using capital letters
3. Sphinx can search case-sensitive and even different endings
Therefore, if you need better model assignment + better search in general, you must at least enable mysql fulltext