BBTF's Jim's Lab Notes Discussion

Jim's Lab Notes
— Site News, Baseball Talk, and a Bunch of Other Stuff

Thursday, May 13, 2010

Server Performance

In preparation for some changes and updates I have in the works, I’ve been spending some time working on server maintenance, configuration tweaks, and the structure of the site’s template system. I hope to speed things up as much as I can. I also hope to address some of the complaints/problems with the site database server. What I need, however, is a little feedback. So, if you would be so kind, please drop in on this thread from time-to-time to let me know how quickly the site is running for you.

Thanks very much,

Jim

Jim Furtado Posted: May 13, 2010 at 03:08 PM | 60 comment(s) Login to Bookmark
Related News:

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

1. Ron Johnson Posted: May 13, 2010 at 07:33 PM (#3531780)

On balance clearly faster to load. Occasional slow moments, but they feel less frequent and not as slow even then.

2. Moses Taylor peacocks peacock Posted: May 13, 2010 at 07:37 PM (#3531787)

Slower than usual. And 2 lengthy crashes so far today.

3. Craig in MN Posted: May 13, 2010 at 07:38 PM (#3531789)

It's been on and off for the past hour or two......slower than Bengie Molina, then faster than Carlos Gomez. It's go-go right now.

4. Bunny Vincennes Posted: May 13, 2010 at 07:45 PM (#3531793)

Its been terrible since the last service upgrade that was maybe or month or so ago, and long outages today.

5. Spivey Posted: May 13, 2010 at 08:02 PM (#3531805)

It's very hit or miss as far as the performance goes any given instant - but it seems like there have been a lot of patches of time where the web site is very slow or down almost every day now.

6. Jim Furtado

Posted: May 13, 2010 at 08:06 PM (#3531810)

I've fixed a few bottlenecks that have been slowing the site down. I also have some fixes for the template system which should speed the site up considerably and should reduce the number of times the page caches need to be build (those moments when a page loads slowly before reloading much faster).

At the moment, I'm now tracking down a faulty line of database code that seems to be causing the database partition to fill up (which is the issue that most often crashes the database server). I've narrowed it down and hope to have it taken care of soon.

Thanks for your patience.

7. Moses Taylor peacocks peacock Posted: May 13, 2010 at 08:07 PM (#3531812)

3 crashes....

Thanks.

8. Ron Johnson Posted: May 13, 2010 at 08:10 PM (#3531815)

#3 Right. Within a few minutes of my first post the server went for a nap. After it came back it seemed far faster than usual.

On balance it seems better to me.

9. Gold Star - just Gold Star Posted: May 13, 2010 at 08:18 PM (#3531831)

Jim, can you give a preview of these changes and updates you have in the works?

10. UCCF Posted: May 13, 2010 at 08:21 PM (#3531835)

Over the past couple of months, I've had a lot of instances where it will take pages a couple of minutes to load (if they load at all). It's been better the past couple of days (or maybe I've just been busier at work and had less time to be here). I've never noticed a pattern for why/when it would happen, except for the usual midnight outage that seems to happen every day.

11. Jim Furtado

Posted: May 13, 2010 at 08:49 PM (#3531883)

FYI, my first priority is to improve site performance and reliability. My second priority is to improve the usability of the site. My third priority is to fix/improve some of the other ongoing issues (i.e., working archives and a more thorough search function). Once those three priorities are taken care of I have a long list of improvements and features (mobile version, Primeys, and other member feature requests that I've received over the last few years). Unlike some of the past upgrades, I'm introducing changes incrementally to minimize the impact on our longtime members.

12. kthejoker Posted: May 13, 2010 at 08:56 PM (#3531893)

Do you want any volunteer IT help with the site? I'm a database administrator and web developer by trade, I'd be glad to assist in any way I can if you've got jobs big or small that can be handled remotely.

13. Bernal Diaz has an angel on his shoulder Posted: May 13, 2010 at 08:58 PM (#3531897)

I want a puppy.

14. Fred Garvin is dead and Joe Biden is alive Posted: May 13, 2010 at 09:00 PM (#3531903)

It's good now, but at times today has sucked like a 99-cent whore.

15. Bernal Diaz has an angel on his shoulder Posted: May 13, 2010 at 09:01 PM (#3531905)

Actually all I really want is a text to post for the lounge. And a puppy.

16. JMPH Posted: May 13, 2010 at 09:13 PM (#3531925)

It'd be nice if the forums didn't require a full refresh to get new posts. Something AJAX-y would be cool, but admittedly a pretty big project.

Also, I want Bernal's puppy.

17. Bunny Vincennes Posted: May 13, 2010 at 09:30 PM (#3531955)

Text to post would be awesome, though I'm getting a blackberry shortly.

18. Pasta-diving Jeter (jmac66) Posted: May 13, 2010 at 09:31 PM (#3531957)

It's very hit or miss--all-or-nothing

19. phredbird

Posted: May 13, 2010 at 09:32 PM (#3531960)

i know i should remember, but sometimes i go to the newsblog before i log in. so if i go to the newsblog, then click on a thread, then finally remember to log in, i'm sent back to the home page and then have to follow the path back. it's not a big deal, but if you can address that it would make visiting this great site that much more enjoyable.

20. JMPH Posted: May 13, 2010 at 09:34 PM (#3531963)

21. Bernal Diaz has an angel on his shoulder Posted: May 13, 2010 at 11:14 PM (#3532083)

Also, I want Bernal's puppy.

Expect a package in 3-5 business days.

22. Andere Richtingen

Posted: May 14, 2010 at 12:35 AM (#3532164)

For the sake of full disclosure, I will point out that Fred Garvin actually is a 99-cent whore.

I have noticed since Jim has been tinkering occasions where the site is behaving much better. Interspersed with periods of horrible performance.

23. Gern Blanston Posted: May 14, 2010 at 12:48 AM (#3532176)

For the sake of full disclosure, I will point out that Fred Garvin actually is a 99-cent whore.

Does that place him above or below me in the filthy whore hierarchy?

24. Vrhovnik Posted: May 14, 2010 at 03:21 AM (#3532354)

Trey Hillman can monitor site performance full time now.

25. Accent Shallow Posted: May 14, 2010 at 03:29 AM (#3532361)

One minor issue I've noticed is that when I open a new tab, it will occasionally have a larger font size than my default. This is mildly irritating, since I either have to refresh the page until it has my default size (this may take a few tries), or change the font size (which makes my other windows have a font smaller than my default). This only happens on BBTF, so I don't think it's a browser issue (I use Firefox, for the record)

26. Fly should without a doubt be number !!!!! Posted: May 14, 2010 at 03:42 AM (#3532375)

Oh, another thing (along the lines of phredbird's 19, which is a great point), sometimes I'll visit the main page when logged in, and it will say I'm not logged in, and won't show me the Lounge or anything. But if I click the logo (to take me to baseballthinkfactory.org/files) I will be logged in. It's bizarre.

That's because of the way this site uses cookies. You have to log in to www.baseballthinkfactory.org separately from baseballthinkfactory.org. Unless you accidentally logged in to baseballprimer.com. Or bbtf.org. Or jimfurtado.net.

27. Dan Szymborski

Posted: May 14, 2010 at 03:42 AM (#3532377)

Mid-day database messages of death are a different matter. Sometimes, if we have to ban a spammer, the database hiccups a bit and the site goes down for a minute.

28. Bunny Vincennes Posted: May 14, 2010 at 04:18 AM (#3532395)

That's probably true but recent outages have been portions of hours, Szym.

29. base ball chick Posted: May 14, 2010 at 05:36 AM (#3532418)

bernal,

i gots a new puppy.

and her name is jana brattain Dog.

and jim,

site seems the same to me - not sure whether or not that is good or bad

30. Avoid running at all times.-S. Paige Posted: May 14, 2010 at 06:17 AM (#3532427)

I often access this site through my blackberry. Not sure if this is a big deal or typical for bb's (since I just got it) but I can't use the edit function after my comment is submitted. I can click on it, but nothing happens. Again, this doesn't strike me as a big deal, but maybe you want to know it's happening.

31. Horror Posted: May 14, 2010 at 06:28 AM (#3532430)

Ditto #12

32. Bunny Vincennes Posted: May 14, 2010 at 06:42 AM (#3532431)

I should also note I get the internet through a connection like an oil well drilled 5000 ft into the Gulf of Mexico. This little nonsense is nothing. And it sits and spins. It is silly.

33. The Keith Law Blog Blah Blah (battlekow) Posted: May 14, 2010 at 07:02 AM (#3532433)

One thing that's been bugging me: the link back to the main newsstand from a specific newsbeat goes to http://www.baseballthinkfactory.org/files/newsstand|lab_notes/, which is a bad link, rather than http://www.baseballthinkfactory.org/files/newsstand.

34. bjhanke Posted: May 14, 2010 at 12:18 PM (#3532461)

I use the site mostly late at night. Since the changes, it has been much faster loading, which was a pleasant surprise. I think late night loading times may have had to do with server and/or database backups. I've done computer backups at night (which is when you have to, to keep traffic flow more or less even), and they can easily shut down a site. In fact, many sites that I know of actually do go dark during backup. BTF never has, but it's never been fast, either. Now, it's much better.

I had exactly one time when I tried to go to BTF and was told the server was down (which is, I think, the message that translates "we're doing backup now"). The interruption was much shorter than usual, and the site was fast when it let me in. All in all, a good job so far.

- Brock Hanke

35. Edmundo got dem ol' Kozma blues again mama

Posted: May 14, 2010 at 12:28 PM (#3532464)

Some sporadicism, but I think my work laptop is getting old and cranky. Like me. Get Bernal and his puppy off my lawn.

36. Jose Can Still Seabiscuit Posted: May 14, 2010 at 12:58 PM (#3532480)

I'll second the comments that yesterday was noteworthy in it's difficulty. Other than that it has been good lately.

To pile on #19 a bit, is there any reason the login can't simply be made a part of the main screen rather than having to click to a separate login page? I don't know squat about page design so this may require more programming than I realize but I've had the same experience as phredbird.

37. Jolly Old St. Nick Done Jumped The Ship

Posted: May 14, 2010 at 01:04 PM (#3532486)

For the most part the site works OK, but there are still two longstanding problems.

---The pages often take forever to load, again on a totally random basis. (And no, it's not my computer or anything else on this end; I have 25 Mbps, which should be plenty fast enough, and this isn't a problem with other sites. Also, it's the same problem whether I'm using Firefox or IE.)

---The font size (very) often changes without rhyme or reason

---You need to post some sort of instructions to deal with cases where all the posts suddenly start appearing in all italics, which happens fairly often. The way it is now, usually about the 10th or 20th person who comes along recognizes the problem and knows how to fix it, but why should we be dependent on that one person? What causes this problem and how do we fix it?

38. Jose Can Still Seabiscuit Posted: May 14, 2010 at 01:08 PM (#3532487)

The font size (very) often changes without rhyme or reason

39. Jolly Old St. Nick Done Jumped The Ship

Posted: May 14, 2010 at 01:26 PM (#3532505)

I have changed from using IE to FireFox about 2 weeks ago. I had NEVER had this problem with IE and now it's happens maybe 5% of the time I refresh (which is often during a Game Chatter) with FireFox. Not sure which you use but it may be a browser-specific thing.

Thanks, but as I also said about the frequent loading time problem, it doesn't matter whether it's Firefox or IE. 95% of the time I use Firefox, but I had the same problem with IE when I used that more often. But that 5% figure you mentioned is about right, and 99% of the time when I open a new thread the problem disappears.

40. The Keith Law Blog Blah Blah (battlekow) Posted: May 14, 2010 at 07:32 PM (#3532989)

You need to post some sort of instructions to deal with cases where all the posts suddenly start appearing in all italics, which happens fairly often. The way it is now, usually about the 10th or 20th person who comes along recognizes the problem and knows how to fix it, but why should we be dependent on that one person? What causes this problem and how do we fix it?

Isn't this just someone leaving an italics tag open? Let's see.

41. The Keith Law Blog Blah Blah (battlekow) Posted: May 14, 2010 at 07:33 PM (#3532993)

Is this in italics?

EDIT: Yes, so let's fix it.

42. The Keith Law Blog Blah Blah (battlekow) Posted: May 14, 2010 at 07:36 PM (#3532996)

Fixed?

EDIT: Yes. So there you go, Andy.

43. Lassus Posted: May 14, 2010 at 07:55 PM (#3533024)

Everything seems pretty much exactly same now as it's been since I joined three or four years ago - no change.

Which... might not really be that great a thing to be noting.

44. Jim Furtado

Posted: May 14, 2010 at 08:06 PM (#3533034)

I've fixed the newsstand link on the beat pages.

I've fixed a scripting error on the newsstand which I hope takes care of the font problem. If you still have this problem please post the url to this page.

The reason a specific page will take longer to load every once in a while (rather than every page loading slower during peak traffic times) is for the caching (storing a full version of a page) the site uses so that every page isn't dynamically created. If you request a page at the time the a page is being regenerated, it will take longer. To reduce this problem I'm reexamining the builds of the site's underlying templates. I'm confident, from some of the trials I've done, I'll be able to improve site/pageload performance.

As for the nightly downtime...backups get done after midnight. The comment table is now 1.2GB. When the database gets backed up, the table get locked. I realize the temporary downtime at that particular moment must be annoying, unless/until I can convert to a different database table type (which will have its own issues), it's something people, unfortunately, will have to tolerate.

45. bobm Posted: May 15, 2010 at 06:26 PM (#3533618)

[30]
I often access this site through my blackberry. Not sure if this is a big deal or typical for bb's (since I just got it) but I can't use the edit function after my comment is submitted. I can click on it, but nothing happens. Again, this doesn't strike me as a big deal, but maybe you want to know it's happening.

Typical in my experience too (BB 8830 on Verizon).

Also wrt Blackberry: is there some reason why there are database outages that are specific to browsing the site using a Blackberry? There are times when, browsing the newsblog from my blackberry, I get a database error message. On a few occasions when I've immediately gone to my desktop after getting such a message, I then have no problem browsing the newsblog. (Sometimes I can bypass the DB error message on my BB by clicking on the newsstand link; I don't know why.)

46. Jim Furtado

Posted: May 16, 2010 at 06:07 PM (#3534142)

The edit function uses Javascript. Does the browser on your Blackberry support Javascript?

I'm not sure why you are getting database errors on your BB.

FYI, in about month I'll finally be getting a new smartphone, so I'll be able to get to work on a mobile version of the site. It's one of the most requested features.

47. Bernal Diaz has an angel on his shoulder Posted: May 16, 2010 at 06:13 PM (#3534145)

No puppy?

48. McCoy Posted: May 16, 2010 at 06:55 PM (#3534165)

If there was a mobile version of this site and BRef I'd never get any work done ever again.

49. Fly should without a doubt be number !!!!! Posted: May 16, 2010 at 10:28 PM (#3534301)

If there was a mobile version of this site and BRef I'd never get any work done ever again.

Because you'd be constantly refreshing to see if the database or server was ever going to come back up?

This site works just fine in Opera Mini, btw. No need for a mobile version.

50. Gold Star - just Gold Star Posted: May 16, 2010 at 10:33 PM (#3534304)

Down for a long while on a Sunday afternoon?

51. Fly should without a doubt be number !!!!! Posted: May 17, 2010 at 11:45 AM (#3534504)

And a monday morning. Also, performance sucks right now.

52. Lassus Posted: May 17, 2010 at 12:18 PM (#3534511)

I get a database error message.

This is currently not uncommon.

53. Jim Furtado

Posted: May 19, 2010 at 01:59 PM (#3536739)

I'm getting significantly improved performance on the server after the fixes. What's everyone else getting?

54. RJ in TO Posted: May 19, 2010 at 02:05 PM (#3536742)

I'm still getting periodic database error messages, but those are now disappearing after a page refresh, rather than lingering. Page load time also seems to be improved.

55. Craig in MN Posted: May 19, 2010 at 02:27 PM (#3536769)

Pages are loaded very quickly now.

If possible, I'm getting more frequent font size changes. I just reloaded pages 6 times, and the font went big, normal, big, normal, big, normal. Now they are staying normal with multiple reloads.

Edit: Though after posting that, the page reloaded with big fonts again, but now reloads stay at normal size.

56. SoSH U at work Posted: May 19, 2010 at 02:31 PM (#3536774)

Did the site shutdown at 1 CDT this morning? I posted a comment which took a little while to load, I looked up and noticed it was a little after 1 a.m., a time the site would often be dark. But the comment eventually got through.

57. Jim Furtado

Posted: May 20, 2010 at 07:17 PM (#3538091)

SoSH. I'm working on the 1 AM site outage. I hope to have the problem fixed (or at least minimized) within the next few days.

58. Dan Szymborski

Posted: May 20, 2010 at 07:58 PM (#3538157)

SoSH. I'm working on the 1 AM site outage. I hope to have the problem fixed (or at least minimized) within the next few days.

Don't tell them it's a bug! Tell them it's a Bedtime Enforcement Feature to keep BTF from sapping the productivity of the legal profession.

59. Rafael Bellylard: Built like a Fielder Posted: May 24, 2010 at 01:41 PM (#3541191)

FWIW, the site is loading slowly for me again (as compared to other sites).

But it's worth the wait :)

60. Harold can be a fun sponge Posted: May 25, 2010 at 12:37 AM (#3541839)

This site works just fine in Opera Mini, btw. No need for a mobile version.

I disagree. Just because a site isn't broken or unusable on a mobile browser doesn't mean that it can't be optimized to be significantly better. For example, putting Hot Topics in a sidebar works great on a regular browser, but less so on a mobile browser (it'd probably make more sense to have tabs where you can switch between reading a thread and looking at Hot Topics). There's also room for improvement when entering posts.

I agree that the site is usable on mobile browsers (I spend plenty of time on it). But I'm glad to hear that Jim's going to try to improve on it, too.

You must be Registered and Logged In to post comments.

<< Back to main

Thursday, May 13, 2010

Server Performance

Reader Comments and Retorts

BBTF Partner

Bookmarks

Hot Topics

Syndicate