Cache Synchronization

[Corfman Clan]

Some of the testers have asked about cache synchronization. Here's some thoughts on what we may implement:

• Have a button (or two) on the cache page to mark a cache to be updated (retrieve the latest logs) or to resync all the logs. Then the cache will be updated through the normal cache update methods.
• Instead of marking the cache to be updated, just update it in real time (this would use your Geocaching.com account).
• When you access the website, check what caches you have found and then update any of those caches that are not up to date. This would use your Geocaching.com account and would be an option you can turn on or off. Updates would be managed automatically.

To start, the above mechanisms would only retrieve find logs for a cache and any other information from the cache page. It would not cause the points for the cache to be recalculated (well maybe, we'll see) and it would not update the leaderboards. Basically, statistics would be updated through the normal process. It would also cause the removal of any archived caches.

You may have noticed that the above does not include a mechanism to add missing caches. That's not quite so simple and so a way to add caches will come later.

[larsslc]

Last Thursday (Sept. 6) I found several caches. Only one showed up in my recent finds list. Friday I found several more caches and again only two showed up. The only reason this causes me any concern is that one of the caches I found on Thursday was a 583 point cache (The Last) and it still shows as being unfound in the Loneliest Cache list. How far away are we from some form of cache synchronization? Thanks for all you have done. You have revitalized my caching again.

[Corfman Clan]

Last Thursday (Sept. 6) I found several caches. Only one showed up in my recent finds list. Friday I found several more caches and again only two showed up. The only reason this causes me any concern is that one of the caches I found on Thursday was a 583 point cache (The Last) and it still shows as being unfound in the Loneliest Cache list. How far away are we from some form of cache synchronization? Thanks for all you have done. You have revitalized my caching again.

Well, unfortunately, we're really about as far away from this as we were when I started this topic on July 17. I will add, however, that it is much closer to the top of the to do list than it was back in mid July.

[Team Opjim]

I have also noticed that caches seem to synchronize in a random fashion: this is mainly a concern when you find one with significant points and it then takes several days to show up on the list. Particularly odd is when caches found yesterday show up before caches from several days ago. In the end, they all seem to show up however.

[Corfman Clan]

I have also noticed that caches seem to synchronize in a random fashion: this is mainly a concern when you find one with significant points and it then takes several days to show up on the list. Particularly odd is when caches found yesterday show up before caches from several days ago. In the end, they all seem to show up however.

As of this morning, there are 103,110 caches in LonelyCache. Do you really expect to see some order on how they are updated with the few you are finding? Each cache is updated about once a week. It's update cycle is it's own and is not based on any other cache's update cycle. A cache may also be included in a pocket query each week. The PQs are used to ensure all caches are included in LonelyCache, not to retrieve find logs for a cache, though that is a side benefit. The PQs are based on Placed Date and State, so new caches will tend to be updated together since they were added to LonelyCache at essentially the same time each week.

I plan to add the last update date/time stamps for a cache on the cache page so everyone can tell when a cache is due for an update. That might ease people's concerns on when they will see their finds showing up.

[Taz]

Ah I was wondering how that was working, after your explaination I now understand. Thanks for taking the time to explain.

[Cache MACS]

After reading everything above, I feel a little better. I am new to the Lonely Cache site and I was concerned for a couple recent finds that were not showing up, because they are higher point caches. Now I know I will just have to be patient.

[Team Opjim]

As of this morning, there are 103,110 caches in LonelyCache. Do you really expect to see some order on how they are updated with the few you are finding? Each cache is updated about once a week. It's update cycle is it's own and is not based on any other cache's update cycle. A cache may also be included in a pocket query each week. The PQs are used to ensure all caches are included in LonelyCache, not to retrieve find logs for a cache, though that is a side benefit. The PQs are based on Placed Date and State, so new caches will tend to be updated together since they were added to LonelyCache at essentially the same time each week.

I plan to add the last update date/time stamps for a cache on the cache page so everyone can tell when a cache is due for an update. That might ease people's concerns on when they will see their finds showing up.

I didn't mean any offense. Now that the system is more familiar I've recognized that it takes a few days for everything to synchronize but it always does.

[TheBlizzles]

I will add a curiosity here ... maybe this will resolve with the next cache update, but still some questions:

I found The Pools on 9/2/12 ... did not log it until ~9/20 (or somewhere where around there). There were finds after me that logged before me.

According to the LC page, the last cache update was 9/21 (I honestly can not remember if I logged before this or not) ... Although there are listed finds on 9/22 and 9/25 (how are finds showing up if the cache has not updated?)

Right now I am wondering if my find does not show up due to some logic quirk where there are finds after mine in the LC database, so it does not look for earlier finds. OR, maybe if it is a roughly weekly schedule for updates, my find will be added today (or soon) when the cache updates ... BTW, I have no problem waiting to see if this takes care of itself in the normal process for debugging purposes.

I also found a cache on 9/1/12 and logged ~9/20, which shows in the LC. The cache was synced on 9/25, but no one found it after me before then. So, not sure if it was the fact that no one has a log showing up after mine before I logged or if just the fact that the cache synced after I know I logged it ... is the reason it is in the LC database.

[TheBlizzles]

Well, nevermind.

The cache just updated a few minutes ago (what time is used? ... must be Eastern Time? Says update was at 11:11AM?) ... or maybe I need to change my setting that was mentioned in the last Release Notes ... (YUP, just found the time was set to Eastern (odd for a default since this is all West coast).

Anyway, my find showed up, so I must have logged just after the update a week ago ... so, the system seems robust to slackers like myself

[Corfman Clan]

According to the LC page, the last cache update was 9/21 (I honestly can not remember if I logged before this or not) ... Although there are listed finds on 9/22 and 9/25 (how are finds showing up if the cache has not updated?)

When logs are received for a cache in a pocket query, we are not marking the cache as updated. We only mark a cache as updated when we use the Live API. The reason for this is that with a pocket query, only the last five logs are returned whereas with the Live API, we receive the last 30 logs (The hope is that no cache will have more than 30 logs from the last update to the next update, though that may occur from time to time). If we mark a cache as updated after a pocket query, then we may never use the Live API to retrieve logs and so we may end up missing logs. This would occur if there were more than five logs between pocket queries.

So in a nutshell, a cache may have logs dated after the last update because a pocket query that incluced that cache was run since the last update.

[Corfman Clan]

The cache just updated a few minutes ago (what time is used? ... must be Eastern Time? Says update was at 11:11AM?) ... or maybe I need to change my setting that was mentioned in the last Release Notes ... (YUP, just found the time was set to Eastern (odd for a default since this is all West coast).

We are defaulting the time zone to the local time of the web servers LonelyCache is running on. These servers are located near Boston, MA and that is why the default time zone is Eastern time. Please do update the time zone for your LonelyCache profile to your local time zone.

[chris geertsen]

yes there needs to be a better system alright. i have found some very reccently and also have hid some. but the ones i hid and the ones i found did not show up on the webpage. like i would find some then some of mine would be published then immediately i would come back to my webpage but they still would not be there. i want to see my satistics updated on a regular bases. i am a little worried because as time passes and there still not showing up, other cachers on the list are passing me up very rapidly. from 2 days ago when i was 777 on the list for utah best all around then to day when i checked i found my self at 786 best all around for utah! so any imformation on how often this gets updated would be very helpfull to me. my user name here is the same as my caching name. or email me at zgeertsen@yahoo.com thanks!

[TheBlizzles]

Everything is updated at least weekly. You can tell when a cache will update next based on the date shown on the caches page toward the top ... add a week. That will be a full update, although partial updates could happen sooner.

[Corfman Clan]

yes there needs to be a better system alright. i have found some very reccently and also have hid some. but the ones i hid and the ones i found did not show up on the webpage...

Hi Chris, welcome to LonelyCache.

As The Blizzles mentioned, every cache should be updated about once a week. New caches are added about once a week too. Currently, for Utah, new caches should be added very early Monday mornings. This means that your latest cache should be added to LonelyCache Oct. 8.

Currently, LonelyCache has 104,049 geocaches. To update every cache weekly means about 15,000 caches need to be updated every day. It's just not practicle to try to update them more frequently.

However, with that said, we will be enhancing LonelyCache to automatically update members finds when the members visit LonelyCache. I am not prepared to offer an estimate of when that will happen.

[Taz]

I like the way Lonely Cache updates. It gets me to check the site on a daily bases to see the changes. If you have the points coming then it's not a big deal to wait for them to show. They always do show up.

[Redfist]

Keep in mind our method contrasted to the method DGP used to use.

Back when DGP was written, there was no official API to get data from groundspeak. Therefore, the approach had to be "scraping HTML" which means having code that looks at the cache pages as you and I see them and grab the pieces of data it wanted. Any time groundspeak changed it's formatting of cache pages, the code had to change. This was also initiated in a manual process. The data was collected "offline" (ie: not directly into a hosted service) initiated by a human and then eventually uploaded to the DGP site. This meant a lot of manual effort on an ongoing basis for years with events like vacation or fatigue getting in the way of "regular updates".

Our method is automatic. We've put the burden of that effort into code hosted online. The system updates itself by calling those APIs and spreading the work load over days since doing ALL caches in ONE day isn't feasible (due to API limits imposed by groundspeak). Even though the updates may appear to trickle in and therefore seem 'partial', be assured that they will make it on a regular ongoing basic. Our system isn't subject to vacations or the eventual personal fatigue that would inevitably affect anyone who had to manually maintain such a system.

[Team Opjim]

There was discussion about having a button that would update a cache. Has there been any further thoughts or activity on this? I understand the practicality of having so many caches that they can't all be updated everyday. However, when I make a find that actually is worth a significant amount of points, it sometimes can take 6 days before that cache shows up on my list, and being impatient and wanting instant gratification I always hope that there might be an alternative. I understand if it isn't feasible, but one can always hope can't they?

[Corfman Clan]

There was discussion about having a button that would update a cache. Has there been any further thoughts or activity on this? I understand the practicality of having so many caches that they can't all be updated everyday. However, when I make a find that actually is worth a significant amount of points, it sometimes can take 6 days before that cache shows up on my list, and being impatient and wanting instant gratification I always hope that there might be an alternative. I understand if it isn't feasible, but one can always hope can't they?

I'm not sure if we'll add an update button to the Geocache page. I think we'll just automatically update your recently found caches when you visit the site instead. Basically, when one visits the site, we'll get a list of caches that person has recently found. For each of those caches, we'll see if we have a log for that cacher for the cache (if it's in LonelyCache). If there is no log, then we'll update the cache.

I've started work on updating the points for a cache right when the cache is updated. Currently the points aren't updated, so if new logs are retrieved for a cache, a message is displayed on the cache page. It also looks funny from when new logs are added to LonelyCache until the points are recalculated when stats are updated. This is especially noticeable for high point caches with few finds (lots of achilles points). I think this is important to complete before allowing users to update caches as the discrepancy would, I think, become even more noticeable.

Also, I've started thinking about this in more depth. That means I'm working on figuring out how to implement it. My hope is that I'll have something available by the new year. This is actually a rather complicated feature so that may not be doable; please realize that hopes are not commitments. There are a lot of things to worry about such as:

• How to handle an update request when a cache is already being updated or is queued to be updated.
• How often to check a cacher's recent finds. How far back do we consider finds recent (Some slackers like TheBlizzles wait two, three, or more weeks before logging )?
• How do we handle a cacher that logs, for example, 2000 finds after making a run along the E.T. Highway?
• How to properly handle a cacher reaching his/her Geocaching Live API cache quota.
• How to propery handle other API errors.
• Etc.

[TheBlizzles]

Hey now! ! !



Ok. Guilty.