4fite.com Blog

I’ve moved tf2.4fite.com off to a new host over the last few days. The transition should be complete, but if you see an ‘offline’ message that means you’re hitting the old web server - your DNS should update soon enough to see the new one.

The new host is basically a virtual server under my control so I’ve tweaked mysql a little to suit my needs, meaning a massive increase in rendering speed for some uncached pages.

At the old host with restricted mysql resources, the completely uncached homepage would sometimes take up to 20 seconds to render. Now it’s more like 200ms on average when uncached, or < 10ms otherwise.

Get what you pay for I guess. I’ll be using the server for a few other things so I can justify bit of a cost increase to myself.

During this process I pruned off data for players that haven’t been updated in 6 months or more. I may start shifting the site more towards updates for members only, period, instead of a mix. I haven’t decided yet.

All this adds up to a faster-responding site which should see updates for members happen on schedule for the forseeable future.

Update: Thank you achievement-grinders for highlighting the futility of taking stats seriously at all but at least with over-time figures the insanely high numbers will drop off soon enough.

Looking at the homepage, you may notice a slight layout change. This is to accomodate an extended players dashboard.

The new dashboard features the previous all-time listings, and the new listings for two-week figures. Since the two-week figures require regular updates, they only show members of the site (this is denoted by the little 4fite icon beside each header).

While the all-time figures are a bit of a novelty, hopefully the two-week figures are a little more relevant for people visiting the site regularly.

In the future I hope to take snapshots of this type of info, perhaps on a monthly basis, and add an award system; so it may display in your profile “Most points earned - April 2008″, for example.

Gold stars can be awarded to the “most improved” players each month

Update: NFS are willing to help with more resources, which is my preferred solution since I’d like to stick with them. I just need to make absolutely sure I’ve tuned all I can tune. I’ll do this over the next week or so, most likely there’ll be interruptions to the site and the update process as I restructure some of it.

My host for this site, and a few others - NearlyFreeSpeech - is pretty good. Paying for only what you use in both storage and bandwidth really keeps the costs down for hosting a bunch of micro sites. Apparently their distributed hosting setup can easily cater for much higher traffic sites, too, so long as the site’s structure itself can cope with it.

I seem to be hitting a bit of a performance brick-wall with them though. The nature of this TF2 site is that it stores and processes quite a chunk of statistical data, even though the resulting output in terms of traffic is not that large. Given a couple of servers of it’s own it would run without breaking a sweat, as the size of the database has yet to reach even 500 MB, with indexes accounting for about 20% of that space. But in the shared environment of NFS hosting, while their web nodes are distributed, there’s only one MySQL process per logical database, and their default configuration means it has fairly limited memory resources to use (for example, the key buffer size is only 16mb, and I’m not able to use InnoDB tables).

Some of the basic queries that compute numbers on-demand, although they only run once an hour or so and are cached thereafter, take anywhere from 1 to 10 seconds depending on how much memory paging the server is having to do - I assume. Whereas the same query on the same data will be more like 150ms when run on my desktop, which has an abundance of memory.

But, in all things, you get what you pay for, I just can’t afford to pay for much more at the moment.

The bandaid solution for this is what I’ll be working on next; to move some of those longer running queries, those that can’t be fixed simply through optimisation due to the server setup, into the update process so the numbers are generated in advance instead of on-demand. This doesn’t remove the problem of a long running query, it simply shifts it to a location where it’s effects will be reduced.

After I’m sure I’ve done all the optimising I can do, I’ll probably approach NFS and raise the issue with them, to see if there’s anything they can do. Failing that, I may have to put feelers out to some people I know, to see if I can get some higher-spec hosting in exchange for… positive karma? that’s a currency, right?

I’m trying out a new change - player avatars are now loaded from the Steam site once, and then cached on 4fite, as opposed to telling your browser to load them from Valve’s servers.

This should mean that old profiles that don’t get updated will continue to have valid avatars even when the source profile’s avatar is changed on the Steam site. Also, it paves the way for some future additions to the site. If this doesn’t put too much of a strain on the storage and bandwidth costs, the change will be optimised, kept in place and applied to group avatars, too.

There’s a handful of invalid profile avatars that cropped up during the transition, but they should be fixed up automatically by future updates.

In other news, I took a trip to Valve last week for a job interview… a major excursion considering I live in Australia. Unfortunately I didn’t get the job, for reasons of experience. Most of my experience so far is in tiny agencies working on small - medium websites. I’d like to thank Valve for the hospitality and opportunity anyway. We should be keeping in touch for future prospects, and working together to improve both of our web offerings.

I made a widget that can be placed on your own site if, for some reason, you wish to display your profile(s) somewhere. This only works for 4fite ‘members’ (your profile is activated for regular updates).

Example:

Read the rest of this entry (link below) to see the HTML code.

(more…)

Observation of the over-time figures I added in recently shows a discrepancy between the per-class and ‘past 2 weeks’ hours figures that are displayed on the Steam site. I tested this with my own profile.

I was at zero hours for the ‘past 2 weeks’ figure. I then played for a while, over a couple of days. My ‘past 2 weeks’ figure now says 3.6 hours, but there was only a 2 hour difference in my summed per-class figures.

There are two explanations I can think of for this.

First is that the ‘past 2 weeks’ figure takes into account time that isn’t spent in a real game (with both a team and class selected). I mention this because I would have spent at least a half-hour running around testing stuff out on my new PC on a local listen server. That, plus other misc times spectating or joining teams.

Second, and more likely, is simply due to rounding of numbers. If you take a fresh account, and you play each one of the nine classes for 0.09 hours (just over 5 minutes), then your time for each class will most likely be 0.0 (assuming Valve round the hours down) but your time played in the ‘past 2 weeks’ would show as 0.8, because Valve have access to the finer-precision numbers, compared to the ones I scrape from steamcommunity.com.

Neither theory accounts for the 1.6 hour difference on its own. So, without further pondering, I can only assume it’s a combination of both.

In conclusion - if you notice a difference between your ‘past 2 weeks’ figure on the steam site, and the two week figure on 4fite (when it’s available), it’s not a bug as far as I know. They may just be counting different things.

Valve added little country flags to their steamcommunity.com profile pages, similar to the ones on 4fite.com, which temporarily broke my processing of your country > province > city selection. All fixed now though.

If anyone reading this knows where I can grab the set of flag graphics that sites tend to use, chime in please. I’ve been making my own from larger flag graphics but I’m sure there’s a premade and coded set floating around, I just couldn’t locate it when I searched.

I could link to the new ones on steamcommunity but I try to limit that to avatars only.

Update: After a little searching using more sensible keywords, I tracked down the apparent home of the flag set. Sweet. They should make their way onto my site soon.

Welcome to the Whirlpool and OCAU people jumping on to the site in the last few days.

I added an extra table to the individual player profile pages which details some of the historical data that’s now being recorded. There’s still only a couple of days recorded, so not much will show at the moment, but give it a few more days and you’ll start seeing weekly numbers in your profile.

You’ll notice that, in addition to how many hours you’ve played over a certain period, you can also tell how many points you’ve earned, and by extrapolation what your PPH for that period was. This is a bit more reflective of how you currently play when compared to your all-time PPH.

Well I think it’s neat

In other news, a new PC for me! Should arrive either Friday or Monday (c’mon Friday, weekend is 4 fite!). Maybe I’ll actually play TF2 now that I’ll get more than 15 - 25 fps during fights. It’s an E6750 with 4gb ram and a 512mb 8800GTS. Yeah, end-of-line CPU… intended to be replaced with a Q9450 when they arrive assuming quad core is worth it by then.

There was a bug in my profile processing code that prevented adding new groups. This caused an error when adding or updating profiles for players with groups that my system didn’t know about yet. The “add profile” feature of the site would have been affected by this too, and probably explained the handful of people that had joined the 4fite steam group but not added their profile to the site. I’ve fixed that one, and added the leftover profiles myself.

Second up was an error when I seeded everyone’s historical data for the first time. I wanted to fix it before it became an issue down the track, but doing so made me flush out everyone’s historicals for the last (and first) 24 hours or so.

Lastly I’ve added a very basic PPH up/down display to player profiles. It will show underneath the “points per hour” on your own player page. Right now, nothing shows there because of me flushing the historical data, but check again in 24 hours or so and you’ll see it. Periods that get compared are one day, week and month.

In the near future I’ll be adding more statistics that are based on recent numbers only (think Steam’s playtime in last two weeks, but more).

Unfortunately I don’t need to create my own stats site to tell me I’ve played zero hours in the past two weeks

Historical data tracking is now in. Key values from each profile update are now time-stamped and archived away for later use, which means at some point in the future I’ll be able to create pretty - ooh, shiny! even - graphs of your PPH and other things as they rise and… well, PPH is the only value that can actually fall, unless I start recording two-week play time.

Only people that add their profile for regular updates will get any benefit from this though since, due to the numbers involved, it could be weeks, or months before the system updates the other crawled profiles again.

In other historical news, apparently it was Australia Day yesterday. Lucky me, that legitimises the day off I was going to take today anyway. Australia Day commemorates the foundation of a British penis colon at Sydney Cunt, according to the Wikipedia article update I was about to make. I would have used it as an excuse to drink all weekend, but that suggests we need an excuse to begin with.

~raw