3rd time I'm introducing myself, right? lol.

today I was just bored and trying to remember old sites I had belonged to...this was the first one I found that's still around that's a good sign I guess lol.

how have you guys been? (if any of you even remember me)

I remember the ID, just not the actual name.

whats up bro?

I'm doin' alright. just bored outa my mind this weekend

Ha, well even though it's so long between times that you post, better than not at all.

Really I like making things for the site and just figure eventually I will think of a way to get it active. I've been spending a huge amount of time adding features to the review system and sports betting system since I started learning some programming to do that. I've actually worked 1,200+ hhours on the review system.

Nice, it looks good. I've kinda let my sites die since I started getting programming projects from my college classes, I was thinking about starting to work on one of them again, but I have IPB 2.3.3 on my site, and so I'm starting to get spammed by bots, gotta find a way to update that first before I get too into it again

It was being spammed like crazy here after I installed the latest patch and then I realized I didn't read the instructions thoroughly and hadn't done one of the steps. lol

It's still fun working on the sites, but I finally took a break the past week or so because I had eben working a lot on the review system and I always come up with new ideas.... so I had to force myself to take a break or I would just be adding things from now on.

I'm going to have to definitely find some people to post at some point or it will be disappointing.

I'm currently trying to find out where in the world sites such as Amazon get photos of products.... where I can use them. Someone on another site was claiming it's against the law to show cd art without permission, but that makes no sense at all. Look how many items are sold on ebay or reviewed on review sites and they show a pic of the product. I mean how can you sale or review a product and then not even show the product you're talking about? I highly doubt it would break anything to show a picture of a product being reviewed!

Anyway yeah I remember your sites. On the gymnastics-related one it was this same way where people just disappeared. It's hard to keep anyone active.

Oh and by the way have you looked at IPB 3? I like it somewhat, but honestly it actually seems slower than 2.3. They claim that you can't judge that because they have other things running there, but all I know is it's slower than their main company site.

Well, technically it is against the law to show it without permission. However, places that sell it commercially are allowed to show the art when they create a deal to sell the item. With regards to sites like Ebay, its not worth it to them to try to prosecute everyone. Not only does it hurt their reputation, but it also would cost them way too much money in legal fees.

Nah, I haven't looked at IPB 3 yet. To be honest, I didn't even know there was a 3.x yet I'll have to check it out though.

Well I don't know whether to use photos or not then. But I see them everywhere. In a lot of reviews recently I saw photos of the products. I doubt that the review sites are paying money to be able to show ana ctual product in its packaging. It justs eems to be done so widespread that I assumed it's ok. Plus just the fact that ebay allows photos at all would make me think it's ok to show pics of items. It just seems weird that there would be an issue with it and I don't want to use them if it's some big deal. I've seen components for sale which are pretty much like ebay also and those are set up to show pics. In the review system I already am not satisfied with how some pages look and it will look worse if I don't put product photos in. ahhhh.

Yeah in IPB 3 they have a reputation system, an option to show who voted for each thing in polls, and eh so little else that I don't even remember it. They totally re-wrote the coding to where it will be a pain updating modifications though. They supposedly made it more efficient, but like I said... it seems slower to me so far when browsing their alpha version.

Heh, well it's got to do with copyrights. As long as you are not using it for profit, they shouldn't have a problem with you, but they do have legal recourse if they so choose. Most people won't deal with it though. But yeah, as long as people don't have to pay for reviews, etc., you shouldn't have to worry about it. Besides, not trying to be rude, but they wouldn't worry about such a small-time operation anyway. They will spend their time and money prosecuting the big guys if anyone

Heh, well that would be a pain to upgrade if they rewrote the source. Granted, it could be more efficient code-wise. But you also have to look at benchmark stats, not how quickly it runs on a server. If the two installs you are comparing are on separate systems, they could operate differently, especially depending on what OS it's running on and what type of processor it is.

Yeah they definitely re-did the source big time. They've given examples, and for example there is no longer a $this->ipsclass.

As far as who they would go after, I am still deciding whether to sell the system though and so a big site could end up with it. Of course I wouldn't be liable for what they decide to use for graphics anyway. I just personally don't see why they would want to keep people from showing it anyway. It's advertising for them. They may not like having a product reviewed badly though, but look at all of the review sites which do use photos of the products. Lately I have seen so many articles and reviews and they almsot always have a photo of the product being discussed.

Don't you use shared hosting when you personally are running forums? Someone from my host claimed that I am getting occasional "mysql database has gone away" errors because "tables are getting large" and that they can't raise their limit much. ok SURELY a site as inactive as this one would not be even CLOSE to having a "large" database. That makes no sense because I see sites with a million posts which I am sure are on shared hosting.

I am not even using over 1-2% of my disc space or bandwidth, so it would be mighty hard to believe that I would be using 100% of DB space! A programmer even told me when I was working on the review system that I should "not worry" how much DB space any of it takes up because db space is "so cheap now" that it's not even a concern!

I sure hope either this support guy is mistaken or I am misunderstanding him. Because if I already have too big of a DB with this horribly low amount of posts, that's ridiculous.

edit: I just deleted more than a whopping 100,000 rows of logs of various types! I hope that helped a lot. But still I always thought you could get a site as big as AMG even on shared hosting. I know Mike used a dedicated server of his own, but I figured 250,000 posts would still be ok on shared.

The thing with the DBs is dependent on your host. You are assigned a specific amount of space for each database by your host (unless you run your own SQL database). Thus, if they only give you a 10mb sql database, that can fill up rather quick. Hell, I have a 100mb database through my host, and I have to prune it each time it gets close to 1,000 posts. As such, if your host does a poor job of managing database space, they may oversell the space and thus, if each person uses 80% of their database space, you could actually fill 100% of the disk space. That is why I like using hosts that have remote DB servers so that the databases are just that....databases, they aren't used for anything else and they tend to be pretty well managed in that case. However, if you have a dedicated server of your own and wish to run an SQL db on it, you are only limited by the size of your hard disk in the machine. This is where your limitation comes from. Thus, I would recommend checking the size of your current database and the limit your host places on db size and if its getting close, consider changing hosts or finding a dedicated SQL server that is much larger.

This may be where IPB 3 comes in. When they claim its more efficient, maybe they mean more SPACE efficient. The way they store it now, there are multiple entries for a single post. (i.e. when a user posts a new poll, it is placed in three tables: topics, posts, polls) As such, it can grow rather quickly. Also, long posts/signatures take up a fair amount of space as well, and as such, can cause database issues. As I'm sure you already know, each character in a post takes up a byte of space. Thus, a 2,048 character-long post takes up 2 KB of disk space. Then, at 1,024 posts, you already have 1 MB of disk space in posts alone, not including members tables that can take up a lot of space, especially if you have custom profile fields added. Now lets add in skins. Each skin you have has an entry in the cache table, which takes up space as well. Every administrative action you take is logged (as you already mentioned) and that takes up space as well. Also, the forums themselves take up space. The more forums you added to your site, they more space on the db it takes up to store the information. You also run into attachments. Each attachment has an entry in the database pointing out which post it belongs to, what the URL to the image/file is, etc. So as you add attachments, you take up space. Furthermore, the more modifications you make to the forum (provided you utilize the forum's db), the more space on your db you take up. Now, I'm not sure exactly how their coded, but I'm sure the addons such as Casino, Betting, and Arcade take up space in the db to store statistics and variables.

So, as you can see, tables can grow large quickly. People who have "industrial"-size forums are invariably run off of a dedicated database so that space is not an issue for them. Your actual disk space and bandwidth for files as provided by your hosting company do not affect your database size. I would recommend contacting your host to see about possibly getting a dedicated SQL server (sometimes quite expensive though if they do allow it) or if its possible to connect to an external SQL server elsewhere (most hosts block all incoming/outgoing communications with servers they are not affiliated with for security purposes) before you go purchase one so you are sure you can use it before spending the money.

Yeah, I just always thought that DB allotment was accounted for in the disc space limit. That's why I kept assuming I was fine.

Casino anr Arcade would normally take up a lot of space, but they are used so infrequently that there aren't many rows being used by them at the moment. My two mods, though, (betting and reviews) are going to be database eaters. Just from me and Aaron (mainly) using the betting system (and very rarely) there are hundreds of rows. For the review system, just the table I am using for some of the ratings is already 450 rows and that is just for the 10-15 reviews that are there.

For some of my review tables I think I could actually drop the id field though becuase I don't think I will actually even need that one.

Now my host claimed my limit is 500mb, yet earlier they said 16mb. I'm so confused. They better straighten it out! And also I saw you can have unlimited DB space if you get hosted by IPS themselves. But then you pay much more than I am paying and you get much less physical space and bandwidth.

Just as a note: me emptying those tables of logs, difference reports for templates, etc.... knocked my DB for THIS site down form 83MB to 41MB. I cut it in half today just by deleting those logs. My zelda site one went from 17MB to 6MB. lol

Yeah, just a heads up, beware of people offering "unlimited" space. There's not such thing as unlimited space. Space is always limited by something...namely size of the hard disk. If you have a 200GB hard drive, you can only have 200GB of space, you can't have unlimited space. So beware. What they really mean is unMETERED, meaning they do not monitor the amount you use.

Don't take this the wrong way, but WTF!?!?! 450 rows for 10-15 reviews? You need to seriously rework that system, man.... For 10-15 reviews, you should have 10-15 rows... Try to get it to 1 row per review, maybe a couple more if you need to, but 30 rows per review is very, very, inefficient and poorly designed :/

Yeah, the logs take up quite a bit of space, gotta manage those a lot.

No. A programmer on IPB's company forums actually told me to change the review setup to its current state. I kept explaining to him that it would, in some cases, mean adding 30 new rows. He said adding 30 new rows was still better than having 1 serialized field in a row with 30 array elements. So it's definitely not poorly designed, unless he was incorrect. But he is a programmer. He said it was poorly designed the other way.

Basically all programmers I have talked to have agreed that when a row has some set of information that spins off of it, it needs to be in a new table rather than squeezed into that same original row.

The reason it takes up so much space is simply that I came up with an enormous feature list I wanted to implement. I (apparently unwisely) listened to this programmer who said "db space is cheap, don't worry how much space anything uses, I would try to make it run as efficiently as possible rather than save on space and it run slowly" so I kept adding and adding and adding features.

The result is that if I do sell it people are going to have to either disable some features or be on decent servers I guess.

But no it should not have the same number of rows as reviews, because here is an example of what I am talking about: I have it where you rate a product in many different areas. In some categories notice I have you rate as many as 30 different things. Well that's 30 rows in a ratings table. Someone else using the system could choose to not even have those detailed ratings or could use less. But part of my original idea for even having this system was to uniquely allowed extremely detailed rating.

It's going to get much worse, frankly, when it starts being used more. Because here are some more examples: For each PRODUCT (so far less rows than reviews at least) if someone adds it to their favorites list, owned list, wish list, and tracking list... that's 4 more rows for each member who does that. I have added commenting, so each comment is obviously a new row. Each deal submit is a new row. Each tracking of a review is a new row. Each request is a new row. Each faq question or answer is a new row.

So see to have the enormous feature list, it obviously has to eat up DB space. But compare to the official Blog system by IPB, for example. I had trackbacks turned on. Well first off there is the blog row, favorites rows, etc... for blog just like for my system. There are other various rows obviously also. And the trackbacks table was a whopping 30,000 rows of SPAM trackbacks. In fact there is actually a spam table and a regular table and so it was like 50,000 rows of trackbacks which all in reality were spam. So this review system is akin to any big component such as blog. It would take longer to build up 50,000 rows in review system than in blog, however.

Oh and also I allow tags for reviews. So there's more rows per review for those too. I put a limit of 5 tags that can be added, though, to try to keep that low. So really the only things that will truly eat a lot of DB space are the rating rows and the comment rows. But almost ANY system has commenting, so that one has obviously almost "got" to be there.

Bottom line: the rating rows are the only area I am worried about. Because it's not ideal having 20, 30, even 50 rows (for cds there is a new row for ratings of each song). But also keep this in mind: number of rows is not the only thing that matters. The number of fields in them and the field types matter.

For instance, 1 row in the members table is probably the same as 50+ review rating rows I would bet. So anyway its not poorly designed, it;s simply that I am ambitious in what features I wanted and I am going to have to allow disabling of various features if I sell it, I am guessing.

Trust me, I have gone through it for months optimizing it. There's not much else that can be done to optimize the DB storage and maintain the features. The only thing I can currently think of is removing the many many fields I ADDED to the members table and making a new table for those. That way those won't have to be wasting space for each SITE member, but only for each REVIEW SYSTEM user. So if someone never does anything in the review system at all then there would not need to be all of these 40 or so fields wasted on them in the members table.

Of course that would make it less ideal when having to retrieve those stats, as it will be in a new table and require a join.

The system is just an extremely complicated system and to get all of these features there has to be DB pushing to the limit. The queries are not ideal either because in some cases there are 4 or 5 joins on a query. However, the programmer told me that it would be better to do it with those joins than to do it the way the original author did. He put ALL review rows, category rows, etc... into global ARRAYS and had to iterate through them on almost every page. So think how bad that would be for a site with 100,000 reviews. Now instead of going through 100,000 rows in an array it has a query with a few joins in some cases. Not perfect, but better than going through countless rows.

If you notice, most of the pages in the system seem to be taking about 0.10 seconds for execution, though. So it doesn't seem bad thus far as far as that.

What it all boils down to though is if you use every feature I am thinking not many average forum users could handle it. I really planned to never sell it anyway though, so who knows.

edit: by the way he also told me to use a lot of indexes. So that's another concern. I am probably using too many, but since everyone seemed to act like DB space was not a big concern, I listened. And obviously db space IS a big concern so I don't know what he was smoking there. Apparently he thought only rich people on dedicated servers would be buying this.

Also, it may sound bad as far as number of rows, but what can you tell from number of rows? Nothing. If those 30 rows disappeared, it would "sound" better to you that there is 1 row, but it would be 30 more fields IN that main row or it would be the serialized row which supposedly actually takes up more space rather than less despite it being 1 field vs. 30 rows. (by the way those 30 rows only have id, review id, rating name, rating score, root, and time. And I most likely can just ditch the id and time rows as there is no direct editing of those rows anyway and the time is never used.)

So then it would drop to about 4 fields in those rows. Also keep in mine most fields are int fields, not text, varchar, etc. Pretty many varchars in the system in various places, but in these rating rows there is 1 single varchar field and the rest are ints. Sometimes I use tinyint, smallint, and mediumint in some of these tables also. So it's been thought out when designing. There is always room for improvement, but this isn't some lazily thought out design. Look at the release dates between versions when I was optimizing. I think I spent 2-3 months solely optimizing then I released then I spent I believe FOUR more months adding new features and continuing to optimize even more.

I need to figure out how to tell how much space each table is taking up int he DB and how much space indexes are using. I never looked closely in phpmyadmin to where you can even tell that info.

Well, if you think about it, having 30 rows of individual columns vs 1 row of multiple columns is actually slower to pull out. You either have to run multiple queries to pull the information from the database or you have to have an iterative loop that loops through each row. So from an execution standpoint, the less rows the better. Also, if you have 30 rows, those rows take up more space in the db because each row takes a certain amount of space by itself anyway. It's like a file on your computer. It's better to have 1 file with 1000 lines then 1000 files with 1 line. The single file will take less space because all of the formatting information embedded in the file is only written once. With the 1000 files, your writing that information 1000, so a lot more space will be taken up in the long run.

P.S. I'm a programmer, too . I'll be graduating in just over 2 1/2 years with a B.S. in Computer Science, and I've been programming for about 7 years already.

Yes, but the point of one of them was that storing serialized data actually takes up more space than storing it as 30 rows. Something about serializing not being a native DB process so it does not handle it well (also when retrieving it is supposedly slower).

Really no option ois very ideal. To have that feature I just have to deal with some of it not being as perfect as I would like. I kept doubting him also and posting that I didn't see how it could be better that way, but since the other programmers there never said hey no that guy's wrong, I assumed it must be true.

It definitely would take forever to go back to that old way. Eh or maybe not.... I'm just thinking off the top of my head. The main components_public file is 17,692 lines though so if I did change it back to another way, it's not as simple as only changing the coding in one area. There are several areas where it's pulled.

I'll just see how it does I guess. I can always change that later. The same way I changed it last time. What I did was write a very small function which took all of the serialized data and inserted it as new rows in the new table. So if I ever decide it was better the other way I would simply rewrite a function in reverse of that. It was only like 15 lines or so of coding.

One thing I am currently worrying about is timeouts. Don't you think I should have it where when updating x rows, it does it in steps such as updating 50 rows, redirect screen, next 50, redirect, etc? I haven't looked into how to even do that, but I thought about it slightly a few days ago and figured I could increment some counts to do it. Do it where when the count gets to 50, (in the foreach loop) redirect it and then have the redirect url have something like count=50 in the url and then that tell it to start with row 51 next time or something.

That's just my initial thought. I can always see how IPB does it in the acp when it uses steps like that.

Dude, if you ever run difference reports, remember to go delete them. I never knew that it stores those as thousands of rows in the DB until you delete the report!

Well, I'm not talking serialized. Serializing data is very bad. I'm saying have multiple columns, of which you can always only pull out the columns you need.

And yes, I would recommend having it do only X number of rows at a time. 500 should be fine, it gets executed fairly quick. But obviously your browser may time out before it gets a result if you do too many rows. I'd recommend about 500. As far as I know, IPB does it this way just by using redirects and passed variables (but not POST variables).

The problem with having them as separate fields is that then I would have to do x number of fields. In other words there would then be a limit on the number of rateable categories someone can add. (30 for example) and also for categories that don't need that 30, they would still have the 30 fields, so it would be using more space for those than it would under the current setup.

As far as actually retrieving them, it's no big deal. It simply gets all rows with category id x or review id x (which are both indexed). It does obviously need a join or new query the way it is now, but usually it's recommended to keep them in the main table (I think) and if you mean having one row in a new table and thirty fields in that table, that would maybe be quicker since it would only be choosing one row, but it would then go back to the fact that it would have a limit on what you can rate etc.

Also think of this other issue that would arise also: on a product page I have it show the average rating from all members who reviewed the product. So as it is now I use AVG(field name). Back when it was serialized I had to select all rows with the category id, unserialize everything, put them in an array, iterate through them, average them with the php. And even if I didn't have them serialized I would still have to select a lot of rows and then make an array of those fields so in the long run I bet it would take even longer that way.

By the way, some people have told me that you should do your calculations with the php when you can, rather than using the queries to do it. Well this programmer kept saying that's flat wrong and that I should use AVG, COUNT, etc... in the queries rather than the php!

By the way, one of my favorite features here is the ajax list of who's viewing the topic. lol I just saw your name gone from here while I was typing this. Not many sites use that feature (which I requested being made) because the guy who made it had improper instructions in several download packages and yet they were different errors in each one and I luckily had old instructions and figured out how to get it working here. Would be resource-intensive on big sites, but I think it's cool.

Heh, that's pretty cool. Anyways, lol, do what you wish We'll see how it works. I'd recommend doing it both ways and seeing which is better (i.e. which has a better trade-off between space and execution time)

Yeah that's a good idea. Although since I added so many features after doing that changeover my backups would be missing a lot of functions. I could toy with a abckup file though and then install this on a test site or the zelda one (might as well be considered a test one there since nobody posts lol) and see what it would do with the same data.

One thing I wish I could do is gret rid of a lot of monthly fields I am using. For a lot of stats I have a total field, monthly field, and then another field where on the first day of each month it puts the monthly total into it for storage. I do that where I will have records of things or could have contests for who submits the most of a particular thing in a month.

I know I could use date() or gmdate() or something instead, but... how do I have it show the current month's total or last month's etc? Is there functionality to do that directly or would I have to use the timestamp to get the current month and then somehow do a timestamp for the last day of that month and one for the first day and make sure timestamps are between it?

Although that also means keeping timestamps for every type of thing. Still much less space than currently though. And also it would get rid of some weird issues I am also having with the current setup.

Another problem if I change that, however, is that on the profiles I would have to do complicated queries to get all of the info, whereas currently all I ahve to do is take the numbers directly from the members table and theya re incremented and decremented upon addition or deletion of reviews, deals, etc etc.

edit: yeah I guess I will get rid of the DB storage of monthly totals. Then I will figure out the best way of getting the totals via php. Sucks though because now I will need to delete an entire task file which I was somewhat proud of writing since it was one of the first things I did which wasn't so much based off of someone else.