Cron jobs or not?

[blackfang]

Players increasing
* experience per hour
* money per hour
* lost energy per hour
* lost health per hour

Would this be better to do with Cron jobs, or would there be a better way to do
this with a simple code?
Personally I've always used Cron jobs, but friends tell me that its a bad way to do it
and that theres an easier way.

[lolninja]

The other way to do it would be to store the last update date as a unix time stamp, then when you need to query this information preform a calculation on the fly, the problem with this method is that it takes processing time to generate, but it also means that you don't have a massive hit every x number of minutes, so its going to be a matter of working out which is better for your game.

Personally I think cron jobs should be more for maintenance stuff, like checking for banned accounts daily etc, for critical calculations they should be done at run time, though I have limited experience within the PBBG field, so take my words with a pinch of salt.

[raestlyn]

In here I posted a timestamp comparing I did plan to use in my next game, but the project is put on hold at the moment. Feel free to use it as how it can be done. There is other ways, but I did find that the easiest.

[jannesiera]

In here I posted a timestamp comparing I did plan to use in my next game, but the project is put on hold at the moment. Feel free to use it as how it can be done. There is other ways, but I did find that the easiest.

Yeah, I think this would be the way to do it.

[Brandon]

Players increasing
* experience per hour
* money per hour
* lost energy per hour
* lost health per hour

Would this be better to do with Cron jobs, or would there be a better way to do
this with a simple code?
Personally I've always used Cron jobs, but friends tell me that its a bad way to do it
and that theres an easier way.

Use time stamps and update based on elapsed time, just like the other people have mentioned previously.  The why in this case is as important as the how.


Avoid cron jobs for this type of update.  Two huge reasons:
1.  You will be updating accounts that may be inactive, therefore increasing server load unnecessarily.
2.  You will be creating a very unbalanced server load, because you are accomplishing a huge task in a small amount of time that you could have accomplished in very small increments.  People that are on your site when a cron job is run may be able to notice the sudden load on the server.

[ranting]

Well the four points blackfang mentioned I would say should be updated for all accounts in the game. So if you want to avoid updating for inactive accounts you will have to setup a way to avoid that no matter which way you choose.

But I can give two reasons why to use cron jobs:
1. You are certain the updates will take place when you want them. If you let a player trigger the update there will always be a risk of time difference. For example, you want some kind of update to happen exactly at midnight. What if no player happens to load the page in question exactly at midnight?
2. With a bigger page load rate the risk of the page being loaded twice at the same time will happen. So you will have to implement some kind of locking mechanism if you want to be totaly sure you don't trigger it twice by accident.

Having said that, I use both ways. Depending on what I want to update, and how important it is that it's updated on an exact moment. For example, increasing everyones money is something I want to happen to all accounts (except on a few occasions) and I want to be sure it's done at certain times. Thus I have it in a cron job. Reviving dead people depends on how long they have been dead. So I don't want it to update all accounts, nor on any certain times. So I use a timestamp-based technique that's triggered when players load certain pages.

[Brandon]

Well the four points blackfang mentioned I would say should be updated for all accounts in the game. So if you want to avoid updating for inactive accounts you will have to setup a way to avoid that no matter which way you choose.

But I can give two reasons why to use cron jobs:
1. You are certain the updates will take place when you want them. If you let a player trigger the update there will always be a risk of time difference. For example, you want some kind of update to happen exactly at midnight. What if no player happens to load the page in question exactly at midnight?
2. With a bigger page load rate the risk of the page being loaded twice at the same time will happen. So you will have to implement some kind of locking mechanism if you want to be totaly sure you don't trigger it twice by accident.

Having said that, I use both ways. Depending on what I want to update, and how important it is that it's updated on an exact moment. For example, increasing everyones money is something I want to happen to all accounts (except on a few occasions) and I want to be sure it's done at certain times. Thus I have it in a cron job. Reviving dead people depends on how long they have been dead. So I don't want it to update all accounts, nor on any certain times. So I use a timestamp-based technique that's triggered when players load certain pages.

To your initial point about inactive accounts, you will not have to do this if you perform the actions with a time lapse, because if the user is logging into the page, it is implied that they are active.  Inactive accounts will not be loading a page, because they are inactive.

To #1 regarding absolute times can ring true in some cases, and that is the only time that cron jobs of this nature should really be used.  Absolute times.  And in cases like this, you should first try to find a way to avoid an absolute time.

And to #2 regarding an accidental double load, this simply does not have to be accounted for.  When you load the page for the first time, that exact time is stored in your database as their last update for their account, so therefore, if they reload the page again it will check for that last update time, and will not perform any action since a certain amount of time would have to elapse before more actions are taken.  For example, if you want an action to only occur once per hour, and they reload the page 5 times in 1 second, only the initial load is going to update this player's information, as the following 5 reloads will check for that updated time, and perform no action since only a few microseconds have passed.


Original Poster:  Just trust that you will be shooting yourself in the foot if you rely on cron jobs to update all of your accounts at once for things that don't NEED to be updated all at once.  You want a balanced server load, and updating all accounts at the same time is the exact opposite of that.  Create a good time lapse function, and you will be gold.

[ranting]

Well what I ment about the inactive accounts would of course only be valid when you want to update something for all accounts in the game, not the single account loading the page. For example you have an economy system where all players get money every hour, even when they are offline. Whether or not you use a timestamp based update or a cron-job, you would want the update to take place on all the players in the game.

Accidental double load is not from the same player, but when two different players does it at the same time. The database transactions takes time, even tho a very small amount of time. But if two players loads a page at more or less the exact same millisecond, they may both make a successful check against the timestamp in the database. It won't happen very often or maybe never, but it can happen. And if it can happen, you need to deal with it.
This of course once again is only valid when you are updating something that should update more than just the single account.

But balancing server load.. let's say you have 5 tasks you want your cron-job to perform on the hour, so you solve it by performing 5 calls to the database. In the grand scheme of things 5 extra calls to the database shouldn't unbalance your server load. If your cron-job does unbalance the server load, then you probably have bigger issues to look at.

[Brandon]

Well what I ment about the inactive accounts would of course only be valid when you want to update something for all accounts in the game, not the single account loading the page. For example you have an economy system where all players get money every hour, even when they are offline. Whether or not you use a timestamp based update or a cron-job, you would want the update to take place on all the players in the game.

Accidental double load is not from the same player, but when two different players does it at the same time. The database transactions takes time, even tho a very small amount of time. But if two players loads a page at more or less the exact same millisecond, they may both make a successful check against the timestamp in the database. It won't happen very often or maybe never, but it can happen. And if it can happen, you need to deal with it.
This of course once again is only valid when you are updating something that should update more than just the single account.

But balancing server load.. let's say you have 5 tasks you want your cron-job to perform on the hour, so you solve it by performing 5 calls to the database. In the grand scheme of things 5 extra calls to the database shouldn't unbalance your server load. If your cron-job does unbalance the server load, then you probably have bigger issues to look at.

I'm not quite sure if you are understanding the concept of a time stamp update, and perhaps it is our explanation of it that is causing this.  The time lapse scheme that we are advising the OP to use is to update ALL accounts, as long as they are active.  When I use the term active, I mean the person has logged in within 30 days or so, NOT active in the sense that they are currently at your site, loading various pages.

The OP wants to be able to update gold/hour, and other hourly type of gains.  We are advising him to do the following:

1.  Store a time stamp for the last time the particular player's hourly gains were calculated.
2.  Evey time the player loads a page, check the time stamp of this last update, and if more than XX minutes/hours have elapsed, update his gold/stats/etc for his hourly gains.
3.  Store a NEW time stamp at this point, to be checked the next time a page is loaded.

The end result is that all accounts will be update in due time.  They will be updated when the player next logs in, essentially.  In a game like the one the OP is mentioning, there is no need to update every single account at the same time.  It is perfectly acceptable to only update the player's account when he next logs in.  It seamlessly accomplishes the same task:  allowing hourly increases/decreases to take place.  This works fine for money, xp, health, resource farming, you name it.

Two different players loading the same check at the same time is not going to have the issues you are talking about because EVERY player has a separate time stamp.  Every player has their own last_update stamp and so there cannot be a conflict between two players.

And back to the server load question, you are suggesting that the OP uses a cron job to update a record for every single user in his database.  Let's say this is done once per hour, and the OP has 10,000 accounts.  You're telling him to update 10,000 accounts with one call, so every hour there will be a period in which this huge database call takes place.  Alternatively, if he used a time lapse method, the record updates would only be occurring when the user logs in or a call is made to the update check.  The result is you will have 10,000 updates run over the course of the hour (if all 10,000 logged in), instead of 10,000 updates run at the same time.  This is the essence of load balancing:  performing an action over a period of time, rather than at an absolute time.  By default, if you are running a cron job to accomplish this, you ARE unbalancing your sever load.  If you looked at a line graph of the load, you would essentially have a line graph that has one huge spike at a particular time, instead of a constant, and consistent stream of updates.

This may not be so bad for a 1-10,000 player database (or maybe it will!).  But you are shooting yourself in the foot because you are not allowing your app to be scalable.  What if your user base increases to 20,000, or 1,000,000?  If you think that updating 1 million records at one time is a logical, efficient approach then you entirely mistaken.

There literally is no visible difference for the players.  The only difference is how much you are taxing your system. 

[ranting]

The end result is that all accounts will be update in due time.

Yes exactly, it's different with in due time and at the same time. The update will not act on all accounts at the same time which I said my argument was about.

It is perfectly acceptable to only update the player's account when he next logs in.

Well I would say that depends on how the game should work and what you want to update. Lets say that you can steal money from a player, 10% of what he has. Now, if the player's money isn't increased in a timely manner when he's offline, the active players doesn't get the gain in money they should have.
Same goes with health. Maybe you want players to heal even if they are offline, so the active players can attack them.

The OP wants to be able to update gold/hour, and other hourly type of gains.  We are advising him to do the following:

Well the OP wrote "Players increasing X per hour"
I wouldn't interpret that as the OP wants the players to have hourely gain updated when they login. But I guess I interpret depending on how I'm used to deal with it, for example some things needs to updated on a timely manner even if the player is offline and other things can be updated when the player is online.

Two different players loading the same check at the same time is not going to have the issues you are talking about because EVERY player has a separate time stamp.  Every player has their own last_update stamp and so there cannot be a conflict between two players.

You will have this issue as soon as more than a single player can trigger something that may update more accounts than just his own. As I wrote, this is only valid when two players are triggering the same update i.e. they are both checking against the same timetamp.
But if what you want to achieve is to update a single players who is online, then you're are very true that you can solve it all by using timestamps.

And back to the server load question, you are suggesting that the OP uses a cron job to update a record for every single user in his database.Let's say this is done once per hour, and the OP has 10,000 accounts.  You're telling him to update 10,000 accounts with one call, so every hour there will be a period in which this huge database call takes place.  Alternatively, if he used a time lapse method, the record updates would only be occurring when the user logs in or a call is made to the update check.  The result is you will have 10,000 updates run over the course of the hour (if all 10,000 logged in), instead of 10,000 updates run at the same time.  This is the essence of load balancing:  performing an action over a period of time, rather than at an absolute time.  By default, if you are running a cron job to accomplish this, you ARE unbalancing your sever load.  If you looked at a line graph of the load, you would essentially have a line graph that has one huge spike at a particular time, instead of a constant, and consistent stream of updates.

I don't think you understood me correctly. Yes, I'm suggeting that he should use a cron job if he wants to update all players at the same time. For example, _every_ player, online or offline, should receive new money every hour. But only if it's necessary. If you have game where it isn't necessary that they are updated at the same time, or updating players who are offline, then no there's no need for cron-jobs and the player can trigger his updated when he's online.
As I wrote, I use both methods, depending on what it is I want to achieve. Updating 10,000 accounts at the same time using a cron-job or using a timestamp based sollution to update the 10,000 accounts at the same time will render the same database load.

What you are suggesting him is he shouldn't use cron-jobs no matter what. What I am suggesting him, is that it depends. Some situations warrents the use of cron-jobs, some doesn't.

[Brandon]

If there is a need to update all accounts because you want offline activity to occur in semi-real time, then sure, a cron-job will be the most obvious solution.  My suggestion to that would be to find a way around this.  Your app will not be properly scalable if you do not.  One such way he could do this is to update the targeted player if he is attacked while offline.  Update the user's stats/money/health before running your fight/steal routine, and you're good.

Back to scalability, it once again becomes an issue as your db grows.  With every new active user, your inefficiency increases because the imbalanced load increases.  You may start off only having to update 100 accounts an hour, but soon that may grow to thousands/millions.  Writing to a database is very taxing, so the goal is to decrease the amount of writes that are made.

If a cron job is run on 10,000 accounts, 1 time per hour, then you are looking at 240,000 updates in a day.

If you accomplish this task with a time lapse, and only update accounts on login/page load, even IF all 10,000 players loaded in a day, you're probably looking at a maximum of maybe 100,000 updates if each player logs in that day AND plays for 10 hours.  Realistically, you're lucky if 20% of your user base logs in and plays more than 1 hour a day, so you're more likely to see numbers like 2,000 * 1, or 2,000 updates per day.  2,000 updates over a streaming 24 hours, vs. 240,000 updates a day, with an imbalanced load occurring at each hourly increment...  I don't think I need to tell you which is better.

If offline updates are needed, that is fine, but don't update every account in your database.  Crons have their uses, but updating every account in your database should be avoided where possible.

[JGadrow]

I just wanted to state that I agree with the points made by Brandon. Also, it was stated in one of his earlier posts that there ARE exceptions which force the use of cron jobs; however, this should ALWAYS be a last-case scenario.

+1 to Brandon for demonstrating a superior understanding of the mechanics of it all

[lolninja]

It is perfectly acceptable to only update the player's account when he next logs in.

Well I would say that depends on how the game should work and what you want to update. Lets say that you can steal money from a player, 10% of what he has. Now, if the player's money isn't increased in a timely manner when he's offline, the active players doesn't get the gain in money they should have.
Same goes with health. Maybe you want players to heal even if they are offline, so the active players can attack them.

Just wanted to query something here, wouldn't it be possible to simply run the resource update script at this point, so say player1 attacks player2, who hasn't logged in for a week, before the update they have 50p in there account, after the update they have £1500.99 which then 10% is stolen, that way inactive players still lose a fair amount of resources, without the monster updates required by the cronned script.

[JGadrow]

Just wanted to query something here, wouldn't it be possible to simply run the resource update script at this point, so say player1 attacks player2, who hasn't logged in for a week, before the update they have 50p in there account, after the update they have £1500.99 which then 10% is stolen, that way inactive players still lose a fair amount of resources, without the monster updates required by the cronned script.

Yes, in fact Brandon already answered this in one of his previous posts.

[ranting]

Just wanted to query something here, wouldn't it be possible to simply run the resource update script at this point, so say player1 attacks player2, who hasn't logged in for a week, before the update they have 50p in there account, after the update they have £1500.99 which then 10% is stolen, that way inactive players still lose a fair amount of resources, without the monster updates required by the cronned script.

Yes of course it may be possible. And if it's possible, then you should. But then of course you have to keep in mind any risks of race conditions.

But I can think of cases where your game design may require another setup.
For example, maybe you have rankings being updated in real-time where your money is part of the ranking. If your money isn't updated for a week then your ranking will be scewed. Maybe you have a team-based game, where part of your updated money goes to the team. It depends. I clearly see the need to have controlled updates by using cron-jobs. But that doesn't mean it should be used all the time.

[Brandon]

Just wanted to query something here, wouldn't it be possible to simply run the resource update script at this point, so say player1 attacks player2, who hasn't logged in for a week, before the update they have 50p in there account, after the update they have £1500.99 which then 10% is stolen, that way inactive players still lose a fair amount of resources, without the monster updates required by the cronned script.

Yes of course it may be possible. And if it's possible, then you should. But then of course you have to keep in mind any risks of race conditions.

But I can think of cases where your game design may require another setup.
For example, maybe you have rankings being updated in real-time where your money is part of the ranking. If your money isn't updated for a week then your ranking will be scewed. Maybe you have a team-based game, where part of your updated money goes to the team. It depends. I clearly see the need to have controlled updates by using cron-jobs. But that doesn't mean it should be used all the time.

Well we have certainly already conceded that cron-jobs have their use.  However, if you have any sort of hope that your game will achieve even moderate success (in terms of # of users), you will want to find a way to code around them for all-inclusive updates.  If you are dead set on a ranking system that relies on them, then minimize the taxing effects, such as only calculating rankings once per day.

[Chris]

If a cron job is run on 10,000 accounts, 1 time per hour, then you are looking at 240,000 updates in a day.

It sounds bad but it is not. You can put 50,000 accounts with 1 hour updates on a cheap $2/month host (tested it, worked without problems ).

What you said makes sense but... to my experience the performance issues were always caused by active players, not by hourly updates, even if cron was very unoptimized. Also bear in mind that some updates can be done in bulk ("UPDATE players SET gold=gold+$constant" is extremelly fast, even for hundreds of thousands players).

I never got any real scalability problem. Let's face it, the problem is to get players, not to make an application that could handle them I wish I had scalability problems...

[Brandon]

If a cron job is run on 10,000 accounts, 1 time per hour, then you are looking at 240,000 updates in a day.

It sounds bad but it is not. You can put 50,000 accounts with 1 hour updates on a cheap $2/month host (tested it, worked without problems ).

What you said makes sense but... to my experience the performance issues were always caused by active players, not by hourly updates, even if cron was very unoptimized. Also bear in mind that some updates can be done in bulk ("UPDATE players SET gold=gold+$constant" is extremelly fast, even for hundreds of thousands players).

I never got any real scalability problem. Let's face it, the problem is to get players, not to make an application that could handle them I wish I had scalability problems...

The OP asked which is better.  I'm not going to give him the resource-ineffective way of accomplishing this task.  Might as well practice good programming techniques now.  It pays dividends later.

[Brandon]

By the way, I just wanted to make mention that there is some merit in what Chris just mentioned.  Cron jobs for all-inclusive updates are easier, and chances are, your game will never get big enough for this to be a problem.  If coding a time lapse system seems like more effort than it is worth, then don't do it.  Do what works for you.  It IS less efficient in terms of resources, but if it is going to allow you to release a finished product, then go for it.

[Chris]

The OP asked which is better.  I'm not going to give him the resource-ineffective way of accomplishing this task.  Might as well practice good programming techniques now.  It pays dividends later.

The OP asked which is better, not which is more efficient. By 'better' I take into account factors like development time needed to implement, further code maintenance difficulty and so on.
Also, you can not take into account only database resource usage, there is also PHP resource usage. By making non cron system you need to put additional php code on some very intense pages (like checks before each attack if the other player needs update) and these will be run almost all the time. On the other hand the cron php is run only once per hour so the php overhaul is almost nonexistant.

I do not want to start argument, but I do not think that making an overly complex system in order to squeeze a tiny bit of performance is a good programming technique. Besides, even if it was a good programming technique it would not be a good game owner technique (and unless stated otherwise I assume that those who post on this forum want a solution that would let them make a successful game rather than learn how to code 'properly', but I might be wrong of course ).

[Brandon]

"Overly complex" and "tiny bit of performance" are not exactly phrases I would coin in regards to this, but again, that all depends on the size of project.  I'll leave it at that. 

[Chris]

"Overly complex" and "tiny bit of performance" are not exactly phrases I would coin in regards to this, but again, that all depends on the size of project.

Yes, I think I might have exagerated this a tiny little bit

[ranting]

If you are dead set on a ranking system that relies on them, then minimize the taxing effects, such as only calculating rankings once per day.

Well, francly I'd rather start out with the functionality I want in the game and then try and work out what the best way is to solve it. I wouldn't want to cut something from the game which I believe would be a very fun addition, just because it can't be solved in the best technical way. If I were to loose 50% of the players of the game, I'd rather sulk a weekend because they thought the game was too laggy .. but if they were to leave because they think my game is boring I'd probably sulk for weeks
But there's a reason the advice "keep it simple" is in many cases a good advice For many coders this certainly applies to using cron-jobs instead of other ways, including me in some cases where I've givven up to create some complex way and instead gone the easy way with a cron-job

[raestlyn]

Wow, I missed a debate where I could have participated

I'm not sure why the timestamp based updating is so hard to understand by some people. Maybe we should start using its another name, Observer based updating.

It is perfectly acceptable to only update the player's account when he next logs in.

Well I would say that depends on how the game should work and what you want to update. Lets say that you can steal money from a player, 10% of what he has. Now, if the player's money isn't increased in a timely manner when he's offline, the active players doesn't get the gain in money they should have.
Same goes with health. Maybe you want players to heal even if they are offline, so the active players can attack them.

This is one of the misunderstandings you have in here. In Observer way the updating of the both accounts will happen before the result is happened. At the moment you choose to attack the opponent, or better yet, when you look at his profile, the updating is done.

The OP wants to be able to update gold/hour, and other hourly type of gains.  We are advising him to do the following:

Well the OP wrote "Players increasing X per hour"
I wouldn't interpret that as the OP wants the players to have hourely gain updated when they login. But I guess I interpret depending on how I'm used to deal with it, for example some things needs to updated on a timely manner even if the player is offline and other things can be updated when the player is online.

Why do you need to update something for a player if no-one sees it? You can do it easily when someone observes the players account. It doesn't matter if the updating happened a millisecond before you saw it, or 2 hours ago, the result is the same.

Two different players loading the same check at the same time is not going to have the issues you are talking about because EVERY player has a separate time stamp.  Every player has their own last_update stamp and so there cannot be a conflict between two players.

You will have this issue as soon as more than a single player can trigger something that may update more accounts than just his own. As I wrote, this is only valid when two players are triggering the same update i.e. they are both checking against the same timetamp.
But if what you want to achieve is to update a single players who is online, then you're are very true that you can solve it all by using timestamps.

InnoDB and row locking anyone?
 You are trying to bring out more complicated situations than there will be in reality. The game I was planning to do and dwas on closed alpha (where the timestamp comparing code snippet I posted earlier was in) updated cities several times in a second when there was more than 4 players online at the same time without any problems. You do know that the stored timestamp isn't always time(), it can be alot different depending what you are producing.

Say you want to give 100 woods a hour to John. Say John logs in after 31 minutes has passed:

Code: (example) [Select]

//query old timestamp etc.
$old_timestamp=0
$time_now=1860  //seconds
$time_passed=$time_now-$old_timestamp//
$amount_multiplier=100/3600 //100/hour, 0.027 per second.
$amount=round($amount_multiplier*$time_passed) //50.22 woods, but we can't produce partials so we round it down to 50.
$new_time=$old_timestamp+round($amount*$amount_multiplier) //1800 is the new timestamp that gets stored, even when 1860 seconds has passed since the last update.

The timestamp that gets stored isn't the current time, so even if the 2 "same time triggered" timestamps can both update the database because the unix timestamp also counts milliseconds as far as I know.


 

[Scion]

yes the observation...or lazy calculation as i like to call it is pretty much the only aproach that you can use where you have resources increasing at a steady rate...

however a cron job ( or eager calculation) that calls a bulk update is really the best way to do a tick bassed update of a single thing.

if you want to do a complicated calculation that requires several selects and updates per thing...then a cron will cause problems as the number of things grows....but it might be the best option in the begining.

the upshot for me is that i will use OO design techniques to ensure that i can switch between eager or lazy calculation as required....how? easy...just make sure every property is a private or protected member of a class accessed via a getter....Ill start with cron bassed calculation since its easy to roll and then if i find i need to switch to lazy calculation i can just insert the calculation where required into the getter.

Sure this means that ill be adding a check to see if something needs to be calculated each time i acess it, but im also clever enough to make it only check that once or better yet to do it as i read the object out of the DB and use polymorphism to inline or exclude the calculation as required....all hidden away behind the object instation.

and while im here making things even more complicated ... I can think of a third way. especially if their is heavy processing for each calculation.... a priority processing queue on a seperate machine...using the command pattern.

At the end of the day each tool has its uses, we just have to remember to be pragmatic and use the best tool for the job.