Forums >> General >> When are comics checked for updates?

When are comics checked for updates?

2 


sniffnoy
Default-avatar
63 posts
So some comics have known regular update schedules, yet the times they are checked seem to be almost entirely unrelated. Are known, regular update schedules accounted for at all? If not, could they be, somehow?
middlerun
Randomsecondpanel
169 posts
I second this. It would be cool if the system kept track of the average time of day that a comic updates and checked more frequently around that time (and I guess less frequently at times when it doesn't usually update, so there's no extra server load). The smaller the standard deviation of update time, the more it can focus update checks.
meeowth
Teto
510 posts
@page_title = "Checking All Comics"

show = true if is_admin? else show = false
t1 = Time.now.ago(60 * 120) #120 min.
t2 = Time.now.ago(3600 * 14)
comics = Comic.find(:all, :conditions => ["`on` = ? and
comic_checked_at <= ? and comic_updated_at <= ?", true, t1, t2], :limit=>20, :order=>"comic_updated_at asc")

In other words, check the comic only if it was last updated more than 14 hours ago and last checked more than 2 hours ago.
sniffnoy
Default-avatar
63 posts
Hm, that's interesting. I didn't realize it was that often. Maybe something like, for comics with known update times, there could be an additional check shortly past that time, and then another hour later? (Hm, this should probably be the suggestions forum, huh? Oops.)

I imagine the hardest part of implementing this would be actually getting lists of known update times. Not sure how to make that doable.
meeowth
Teto
510 posts
Here is the thing: if a comic really updated at the same time every day, then it would be flagged as updated at the same time every day, since there are checks approximately 24 hours apart.

If a totally consistent comic is always being checked and updated something like 1.5 hours after it really is, then an admin can come along and manually force an update check a little sooner after the real update and then the 24 hour regularity of the update system will take care of the rest.
sniffnoy
Default-avatar
63 posts
The problem is, SFAICT, the actual time between checks is longer than the 2 hours it's supposed to be (don't ask me why, this is just an empirical observation), and while certainly 24=14 (mod 2), you don't have 24=14 (mod 2+e) for most e. So counting on the stability of that doesn't seem too reliable. And even if we assume stability, if a comic happens to be late one night and goes up 1:50 past its usual update time, it'll throw things off for the future if there are no further deviations. (There's also the where-does-this-come-from-originally problem - how do you ensure that the initial check is close to the actual update time modulo 2 hours?)
meeowth
Teto
510 posts
Besides the runner that is constantly checking batches of comics that meet the criteria for being checked, an administrator can force a comic to be checked for updates at any arbitrary time.

Still, minimizing e might be the best solution to minimizing the delay time.

What's the regexp that is essentially the more efficient version of "(.*?)"? We could have ron do a search and replace on that in the database.
corruption
Users%2fcorruption%2fthumb%2favatar
715 posts
When you find out, let me know, and I'll try it on the Smack Jeeves comics in the suggested page.
Celtic Minstrel
Magicharp
267 posts
Um, what? I don't think there is a more efficient version of .*?
corruption
Users%2fcorruption%2fthumb%2favatar
715 posts
Oh well, I did most of the Smack Jeeves comics two days ago.
meeowth
Teto
510 posts
Well, for example, .+? is oh-so-slightly more efficient than .*?, but that isn't the one I'm thinking of. I'm pretty sure what I'm thinking of had ^ in it.

The thing about .*? is that Google doesn't search for symbols... unless I'm missing out on something.

BEGINNING([^END]+) is an example of something even more efficient, but it isn't something that can be dropped in where .*? is now (assuming BEGINNING(.*?)END).
Celtic Minstrel
Magicharp
267 posts
I guess you mean something like [^x]+x ?
meeowth
Teto
510 posts
In this case x changes depending on what the terminating string is, unless there is something I'm missing. (I'm no regexp expert)
middlerun
Randomsecondpanel
169 posts
meeowth wrote:
(I'm no regexp expert)



The correct term is "regexpert".
Celtic Minstrel
Magicharp
267 posts
meeowth wrote:
In this case x changes depending on what the terminating string is, unless there is something I'm missing. (I'm no regexp expert)

If you're referring to my example, it means "match one or more occurrences of a character other than x, followed by an x". If you're referring to something else, please post the case so I can see it.

I dunno if I'm a regex expert, but I do use it fairly regularly.
meeowth
Teto
510 posts
It's possible we're talking about the same thing, but I'm no regexpert. I'm just trying to think of something efficient that can be used as a drop-in replacement for (.*?).
Celtic Minstrel
Magicharp
267 posts
Well, I think it would depend on the context. If you're trying to make it more efficient, it would probably have to be done on a case-by-case basis, unless of course all of your (.*?) are within quotes?
corruption
Users%2fcorruption%2fthumb%2favatar
715 posts
Nope. Trust me, they are not.

I am doing most of the suggested comics and problem fixes lately, and I can tell you that having a general purpose wildcard like (.*?) makes things easier.

Although having a listing for the main Wordpress and Comicpress comics reg_exp /comics/(.*?)" instead of just under Comic Genesis would be a little easier.
meeowth
Teto
510 posts
corruption wrote:

Although having a listing for the main Wordpress and Comicpress comics reg_exp /comics/(.*?)" instead of just under Comic Genesis would be a little easier.

Explain.
I mean, I could change the other methods to be the same as the CG method, but I don't see how that would help.

Messing around with what regexp the preset methods use is pretty easy, though renaming them would fall under the domain of Things Ron Does.

Just two weeks until the university term ends, I can try to mess with the site code a bit more then.
corruption
Users%2fcorruption%2fthumb%2favatar
715 posts
Wordpress, Comicpress and Comic Genesis all can have the reg_exp /comics/(.*?)" used on them.

However, I am thinking that we may have a seperate premade one for Wordpress and Comicpress, without affecting the Comic Genesis one. After all, if Comic Genesis changed it's layout, we would want to change it's reg_exp to something that would work WITHOUT affecting the reg_exp for Wordpress/Comicpress.

Basically, just add another premade reg_exp for those two and don't touch the Comic Genesis reg_exp
2 


Forums >> General >> When are comics checked for updates?


Insert link to comic