feature requests

The larger we get...the slower we get (Comic Book Forums)

  • The larger we get...the slower we get

Author Discussion
  • Posted: April 10, 2012 9:58:44 am
  • So we have had some brief downtime the last few days and I have been able to figure out that it is because people have thousands of comics in a given pile. These queries are taking super long to run (and lock the database up).

    When the site was built it wasn't built with the mindset that people wouldn't create piles, and that piles would be treated more like boxes. One individuals collection has the following:

    Master Pile 5,158
    Completed Sets 4,668
    One-Shots 343
    Grade and Price 35
    Possibly Sold 53
    Robotech And Anime 265

    Another one has this:

    Master Pile 53,429

    If anyone clicks on these large piles the database essentially locks up for everyone else. If the site is really busy that means that it can overwhelm everything else and essentially shut the site down.

    Do I make it where if you share your collection piles that have more than X comics will not be shared? Do I make it where you cannot enter more than X comics into a given pile? Do I make it where piles are not shared, and instead just the publisher list and the series list are (this might be the best bet)?

    What are everyone's thoughts on this?

    Last edited April 10, 2012 10:02:56 am
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: April 10, 2012 11:12:39 am
  • i admit, i am guilty of this.
    i am reorganizing everything so i threw it all into the master pile until i get my boxes straightened out.
    i will get a move on fixing this.
  • Posted: April 10, 2012 11:26:54 am
  • I am guilty of this aswell, i have just started adding my collection to the site and used the master pile as a catch all. I will def start moving them into Boxes
  • Posted: April 10, 2012 11:53:16 am
  • Maybe put a ceiling limit on the amount that someone can put into a pile? If they hit that limit then they would have to create a new pile?
    Image
  • Posted: April 10, 2012 12:09:14 pm
  • Pleading guilty to this situation, too...Embarassedwill do my best to clean up & organize "my realm" better...Angel

    puinno said:
    Maybe put a ceiling limit on the amount that someone can put into a pile? If they hit that limit then they would have to create a new pile?


    Like this idea...Cool
    Pick-ups this month
    ?

    On the way...
    The Avengers (Vol. 1) #58
    Silver Surfer (Vol. 1) #2
    Crisis on Infinite Earths DC/Wal-Mart 100-pg Giant


    Submitted/Back from CGC...
    ?

  • Posted: April 10, 2012 12:39:55 pm
  • My problem with that is what is a "max"? i.e. you want all of your ASM's in one pile, etc.
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: April 10, 2012 12:51:11 pm
  • So you're saying more piles are better as long as they're smaller?
    If so then you would still need a pretty high number for say a box that has All Superman or All X-men.
    As of now I have mine listed in a 1 pile/collection.
    I've been thinking of going though my collection and resorting it according to title...just not looking forward to it.
    As for the Max...guess that is for you to decide, based on what you think the system can handle without freezing up. Once you decide on a number then it will be up to us to break down our collections to fit it.


    Last edited April 10, 2012 1:16:53 pm
    "Disliking everything is not the same thing as having an opinion"

  • Posted: April 10, 2012 1:22:15 pm
  • No matter what the original idea was, it seems intuitive that these piles are just ways to organize your collection in whatever way you want. So if someone has 30,000 Marvel comics and wants that to be a pile they should be allowed to do so.

    So if the problem is then that OTHER people looking at that slows things down, then I think your "best bet" option is the best bet. If I look at Joe Smith's collection I shouldn't need to know how he organized them ... and in fact that would make it harder for me to see if he had a specific comic.
  • Posted: April 10, 2012 1:27:26 pm
  • Yeah the simplest way would be not to let other people view your collection. If you want to sell something put it in a sell pile for buyers to view.
    "Disliking everything is not the same thing as having an opinion"

  • Posted: April 10, 2012 2:21:52 pm
  • abysslord said:
    So if the problem is then that OTHER people looking at that slows things down, then I think your "best bet" option is the best bet. If I look at Joe Smith's collection I shouldn't need to know how he organized them ... and in fact that would make it harder for me to see if he had a specific comic.
    I have already done this...made it where your piles are shown (so people can see how you sorted your collection, but they cannot click on them). They can only view by Publisher/Series.

    Of course the problem is how do you now tell someone check out my X pile as it is for sale if they can't click on the links anymore?

    As far as grouping by publisher, there is already reporting for that...as well as by series. The piles should be a general representation of where the book is located.

    Last edited April 10, 2012 2:25:54 pm
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: April 10, 2012 2:34:10 pm
  • well if they say check the x pile because it is for sale..then they should be making that a sell pile instead of just a normal pile anyways Think
    Image
  • Posted: April 10, 2012 2:45:09 pm
  • I seem to have forgotten what it looked like when you click on "My Collection" Smile

    Ok, after viewing it again .... I think it's fine how it is now. If the only problem is people can't say "Check out my "Sell Pile", I think that's okay.

    Plus, couldn't someone just make their collection private except for the "Sell Pile" and then that's the only one that would show up if someone looks at their collection by publisher/series?
  • Posted: April 10, 2012 5:22:03 pm
  • True, that is an option.
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: April 10, 2012 11:10:44 pm
  • abysslord said:
    Plus, couldn't someone just make their collection private except for the "Sell Pile" and then that's the only one that would show up if someone looks at their collection by publisher/series?


    I really like this option. You could also make the "Master Pile" available for viewing if someone just wants to showcase their new pick-ups or their pride and joys. But for this to work, wouldn't the "Sell Piles" and the "Master Pile" need to have a ceiling on them, otherwise you'd be back to where we are now, right?
  • Posted: April 10, 2012 11:40:02 pm
  • glh08 said:
    abysslord said:
    Plus, couldn't someone just make their collection private except for the "Sell Pile" and then that's the only one that would show up if someone looks at their collection by publisher/series?


    I really like this option. You could also make the "Master Pile" available for viewing if someone just wants to showcase their new pick-ups or their pride and joys. But for this to work, wouldn't the "Sell Piles" and the "Master Pile" need to have a ceiling on them, otherwise you'd be back to where we are now, right?


    Not really because they aren't viewing the pile. When I say "Sell Pile" i mean whatever pile they created for their sell items, not the "Sell Pile" you get automatically. So let's say they want to sell everything in "X-Men Pile". If you look at their piles you can't click on them. If you click on their "series" or "publishers" you get ALL their stuff they're sharing.

    So in this case the person would just have to make everything private but the X-Men pile, and then you could view by publisher or series all the books they're selling.
  • Posted: April 11, 2012 8:31:41 am
  • What if you make it so that if people have a certain number of books in their pile (their ceiling or max). If they go over that number then the pile is automatically private or non-shareable. We can just put a small notification so people know that others cannot see that pile anymore.
    Image
  • Posted: April 11, 2012 8:51:18 am
  • puinno said:
    What if you make it so that if people have a certain number of books in their pile (their ceiling or max). If they go over that number then the pile is automatically private or non-shareable. We can just put a small notification so people know that others cannot see that pile anymore.
    That might work too...I thought about making the links work again for say piles with less than 1000 items.
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: April 14, 2012 11:56:35 am
  • Maximum pile size sounds good, leaves plenty of room for books plus resolves the issues.

    I am probably non-typical in that most of mine are split out roughly in piles by boxes, such as Batman 1, Batman 2, etc. It makes it easier to check something and also to add a new purchase to the correct box. For example, my Batman 1 box might be set for Batman 1 through 175, box 2 176 through 300, etc.

    I think I have about 50 different piles. Shocked
    The joy is in the journey, not just the destination.
  • Posted: April 14, 2012 9:42:25 pm
  • I agree with Spa say each pile has a 300 limit. My long boxes range anywhere from 206 to 272 and the 272 is tight when they are all bagged and boarded.

  • Posted: April 16, 2012 2:56:49 pm
  • dough boy said:
    puinno said:
    What if you make it so that if people have a certain number of books in their pile (their ceiling or max). If they go over that number then the pile is automatically private or non-shareable. We can just put a small notification so people know that others cannot see that pile anymore.
    That might work too...I thought about making the links work again for say piles with less than 1000 items.


    I like this idea, but whatever works for you is okay...just as long as you decide before I start moving stuff around, because I really don't want to do it more than once.Wink
    "Disliking everything is not the same thing as having an opinion"

  • Posted: April 16, 2012 4:08:55 pm
  • I don't like the limit on piles, especially 300. I like to categorize my comics in other ways, such as all Spider-Man in one pile ... which is way more than 300. it would be a giant pain to look at my collection by looking at 4+ different piles.

    Last edited April 16, 2012 4:09:37 pm
  • Posted: April 16, 2012 4:51:49 pm
  • Personally I don't care if people can view my collection...so the way it is now is fine. If people want other to be able to view there collection then they can break it down to what ever parameters DB wants. I was just saying if he's going to make it mandatory then I would like to know so I can start the changes necessary. If I have the option then I would leave as is and not have people see my collection.
    "Disliking everything is not the same thing as having an opinion"

  • Posted: April 16, 2012 9:06:21 pm
  • Abyss, If you do a all pile search it would list all of your Spider Man for every pile on one page or if you did not log in and just searched your screen name you could go under Publisher and view Marvel then Spider Man.

    I hope this helped.
  • Posted: April 17, 2012 7:13:18 am
  • I haven't seen any problems since I first identified what the culprit was and removed the ability for people to click on piles.

    I think I will open it back up for piles that have less than 1,000 items in them.
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: April 17, 2012 9:19:08 am
  • I think a 1,000 issue limit sounds reasonable. I only have one pile with more than 500 comics and I don't feel that I fragment my collection too much at all. I know I don't have the largest collection around here but it is not small either. I use publishers as a first way to split and then I use characters or titles to split the collection further. My biggest pile is Image because I have only removed Spawn from that. Marvel is split into 13 piles.

    Master Pile 0
    Dark Horse 92
    DC 262
    DC Wildstorm 376
    Image 650
    Image Spawn 324
    Misc Other 56
    Marvel 370
    Marvel Avengers 302
    Marvel Daredevil 201
    Marvel Deadpool 299
    Marvel Iron Man 202
    Marvel Punisher 153
    Marvel Ultimate 348
    Marvel Wolverine 320
    Marvel Spider-man 420
    Marvel Amazing Spider-man 306
    Marvel X-Men 276
    Marvel X-Titles 420
    Marvel Uncanny X-men 484
    CGC 25

    If a pile gets too big I just find something in it that stands out and make a pile for it on its own.
    For example: In the start I only had one pile for X-Titles. Then I found that I had too many Uncanny X-Men and I made a pile for this. Then
    I found the title X-Men was filling too much in the pile and I made a separate for this one as well. The same with Punisher. In the start I just put this in the Marvel Pile but when that got too big I saw that Punisher could be a pile on its own.

    I now this doesn't bring anything to the current situation but I thought others might wanted to have some insight into how they could divide thier piles.
    Not collecting any more but I still enjoy comics now and then...
  • Posted: April 17, 2012 9:24:07 am
  • I just went to my collection because I wanted to bring down the amount of comics in the Image pile as this was the largest one. I created a pile named Image Darkness but I don't remember it being so difficult to move comics from one pile to another. Am I doing something wrong or do I really have to press edit and then select a pile for each comic I want to move?

    I know there has been a much easier way to do this but I can remember where?
    Not collecting any more but I still enjoy comics now and then...
  • Posted: April 17, 2012 9:30:03 am
  • You click edit once for each series, and then you should be able to change the pile for each series.
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: April 17, 2012 9:32:00 am
  • dough boy said:
    I haven't seen any problems since I first identified what the culprit was and removed the ability for people to click on piles.

    I think I will open it back up for piles that have less than 1,000 items in them.


    I only have one pile approaching that number (the master pile) so this is no prob for me Whistle
    Image
  • Posted: April 17, 2012 9:39:12 am
  • dough boy said:
    You click edit once for each series, and then you should be able to change the pile for each series.


    That's it... I really couldn't find it LOL. Thanks man Very Happy
    Not collecting any more but I still enjoy comics now and then...
  • Posted: April 20, 2012 10:59:52 pm
  • abysslord said:
    I don't like the limit on piles, especially 300. I like to categorize my comics in other ways, such as all Spider-Man in one pile ... which is way more than 300. it would be a giant pain to look at my collection by looking at 4+ different piles.


    Abyss, I do the same thing, even if my piles are split up. In the Search feature on My Realm page, I can key in Spider man and see every issue I have, broken out by the title. Usually I do this when I know I have a copy but may have a chance to upgrade a grade on an issue that may no longer be on my want list.
    The joy is in the journey, not just the destination.
  • Posted: April 21, 2012 12:08:50 am
  • Spa-fon said:
    abysslord said:
    I don't like the limit on piles, especially 300. I like to categorize my comics in other ways, such as all Spider-Man in one pile ... which is way more than 300. it would be a giant pain to look at my collection by looking at 4+ different piles.


    Abyss, I do the same thing, even if my piles are split up. In the Search feature on My Realm page, I can key in Spider man and see every issue I have, broken out by the title. Usually I do this when I know I have a copy but may have a chance to upgrade a grade on an issue that may no longer be on my want list.


    Yeah, I guess I never saw that 'search' before .... . I guess I've never needed either though, but it's nice to know about..... especially with all the spider titles Smile
  • Posted: May 30, 2012 7:22:14 pm
  • Okay, so I am one of the offenders who has 7000+ books listed in the "Master" list.

    I have started to break it down by location of the books, which should result in 18 separate piles of comics.

    An observation - When I had all my books in one pile, I had a total for amount invested and "value" of the entire collection.

    Now that I have broken them down, I only get a total for each pile at the bottom of the pile. It would be nice if the total amount invested was listed under the "My Realm" page.

    Either that, or a custom report where you list the entire amount "invested" along with value and potential profit. It would be REALLY great if this number could be tracked over time, thus allowing you to get a percentage rate of return.

    Ultimately, I'm not sure why I even want this, as I never sell anything. But SOMEDAY (probably after I am dead), someone might want to see this information to get an idea of whether or not to sell off my collection.

  • Posted: May 30, 2012 7:26:42 pm
  • I am not sure I am following. The "Master Pile" and "Custom Piles" is the same code so when looking at one or the other you should see the same data.

    There isn't really any room to show the "invested" or "profit" on the "My Realm" page. However, you can get this number by looking at your graphs.

    Might I suggest you make this a "feature request" and others can vote it up/down, etc?
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: May 30, 2012 7:47:44 pm
  • Putting a Max Number per Pile will only force others and me alike to create more and more piles and a mess would ensue.

    Myself I have started creating Piles for each Publisher and if a Series has more than 1 publisher then i create a pile for the title in question (i.e.) Star Trek, Star Wars, Transformers and so on.
  • Posted: May 31, 2012 6:54:30 am
  • I think the issue is that one large pile is more difficult for the software to process than many small piles. By breaking up a large database into several small ones, you save the search engine from having to go through a bunch of records that you are not interested in, thus making it faster and more efficient.

    It may be more of a difficulty for the end user to decided how to break up the data into a format that makes sense to them, but it is easier on the search engine.

    At least that is how I interpret what dough boy is saying.

  • Posted: May 31, 2012 6:59:31 am
  • Yes that is what I am saying. I am not going to limit how many you can put into a pile...but I will limit how many a user can request at one time. The only problem area is when someone views your collection. I will look into reworking the SQL to see if I can get it so it doesn't matter how many are in a pile.
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: May 31, 2012 8:49:58 am
  • I have fixed my piles. But I would like to be able to see other's piles, for example Weinreich has a CGC pile and I can't see what is in the pile.
    Latest additions
    Avengers #31, #46, #99, #145-159, #161-164, #197, #200, Annual #6
    Sgt. Fury and His Howling Commandos #7
    Forbidden Worlds #76
    Strange Tales #118, #124, #133, #139, #140
    Journey Into Mystery #125
  • Posted: June 9, 2012 12:03:37 pm
  • All of the recommendations are but no matter what adjustments are made there will always be issues, this sight is fantastic.
  • Posted: July 12, 2012 11:12:56 am
  • I feel DB has solved it. No one has to change anything if they don't want to. I have placed over 6700 into my "marvel" pile and am thinking it could take me a long time to resort all those. Have to admit I am sorta looking forward to the challenge though.
  • Posted: July 12, 2012 11:29:38 am
  • I sorted mine (7000+ issues) and it only took me an hour or so.
    Just create the folders/piles you want ahead of time. (I did Publishers)
    Then open up your collection and just open the drop box, then "edit tile" and change (set all piles to) the new pile and all the issues will change, it only allows a few hundred issues to be moved at a time so you'll have to scroll to the bottom and save every once and a while.
    Worked slick. I was worried it'd take me a few weeks but it didn't.

    Last edited July 12, 2012 11:33:54 am
    "Disliking everything is not the same thing as having an opinion"

  • Posted: July 12, 2012 11:47:34 am
  • The only time we were having performance issues is when you looked at someone else's large pile and it was comparing against yours. Since I stopped that we have not had any downtime (and actually over the last 2 months only 4 minutes of downtime which works out to like 4 seconds a day. I still don't think we even had the 4 minutes and it was likely a reporting issue.
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: January 7, 2014 7:52:36 am
  • I am trying to create new piles to move things around. I have followed the directions. If I create a new pile a box next to it shows up to edit, but my master piles does get the box. I have opened the share to yes. I went to the recommended site in the forum and followed the directions in it. But it won't let me. Any idea what I am doing wrong. I am trying to follow the rules and keep piles to 1000.
    thanks
  • Posted: January 7, 2014 7:55:16 am
  • I am having a hard time following you. When you create a new pile it is essentially a new container to add comics to. They can only be in one container at a time. So you would need to move comics from your Master to your new one...
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
  • Posted: January 7, 2014 8:09:25 am
  • okay so here is the issue I am having. To move comics from the master pile ,maybe I am reading them wrong. To move from one pile to the next you need to enable sharing. Then you go to manage. Then you click on the edit button next to the pile. I am not seeing a edit button next to my master pile which has all my comics. The file I created to move some into, has no comics, but the edit button is there. Am I doing this wrong?
  • Posted: January 7, 2014 8:14:26 am
  • also this is another odd question. I see you can only have a comic in one pile. Say you have multiple copies and you want to sell some copies, ie the multiples but want to keep one. Can you put some in a sell pile and others in a regular pile? Maybe this has been answered if so sorry
  • Posted: January 7, 2014 8:19:03 am
  • Okay ignore my posts, I am a noob and figure it out you can delete them sorry to bother you
  • Posted: January 7, 2014 10:01:44 am
  • Create a new folder with a name ie...Marvel.
    Now go into your Master Pile, Now clickon the + on the series you wish to move to the new pile (marvel)
    drops down, now clicl on the edit folder next to the red X. Now you'll see "Set All Piles to..." there should now be a "Marvel" or whatever you named the new folder. Change the issues or all piles to the new folder.
    hope it helps.


    Last edited January 7, 2014 10:02:38 am
    "Disliking everything is not the same thing as having an opinion"

  • Posted: January 7, 2014 11:20:47 am
  • thanks
  • Posted: May 11, 2014 6:00:41 pm
  • FYI I found the easiest for me to list piles is to match my storage boxes. For example Spider-Man 1 Pile has Amazing Spider-man 1-250 and my long box has the same. AS 2 has 251-500 and a matching box. AS 3 may have Spider-man 1-100 and others.

    I have 6 boxes of Batman title, some are Golden Age short boxes, the rest are long boxes. Some boxes may have multiple titles with shorter runs, such as EC Horror. But I try to match each box with a Pile, so no boxes should have over 300 or so individual books.

    If I want to see all my Spider-man books together (and the CGC books are necessarily in separate boxes), I can just search for Spider-man and them all in order!

    The joy is in the journey, not just the destination.
  • Posted: May 11, 2014 6:12:54 pm
  • Spa-

    Have you added me to your will yet?
    Favorite Quote: "You're not just some guy in a bat costume are you? ARE YOU FREAKING KIDDING ME?! - Justice League #1, 2011"

    Hunting for (mainly reprints/variants) - Please help! - See My Profile
50 posts • Page 1 of 5
World Community Grid Logo
ComicBookRealm.com: 72 years, 255 days, 20 hours of Run Time
Help projects like: Smash Childhood Cancer, OpenZika, Help Stop TB, FightAIDS@Home - Phase 2, Outsmart Ebola Together, Mapping Cancer Markers, FightAIDS@Home
Join World Community Grid today!
When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
  • Newest
  • Al Barrett's Atomman Special # 10
  • Amber Huff's Bad Blood Issue nn
  • Marvel Comics's Ultimate Black Panther Issue # 2 - 2nd print b
  • Marvel Comics's Venom: Dark Origin Issue # 2