Catalog very slow

AfterShot Pro General Questions & Getting Started Forum
Post Reply
tylmaster
Posts: 7
Joined: Mon Aug 12, 2013 2:08 pm
operating_system: Linux
System_Drive: N/A
32bit or 64bit: 64 Bit

Catalog very slow

Post by tylmaster »

I have imported about 90'000 photos into one catalog, and this seems to break my system down.

Expanding the meta tree at root layer takes me more than half a minute, which is not very funny to work... (in clear words: unacceptable!)
Browsing the catalog is also everything else than fast, also saving metadata/selections etc. to multiple images at the same time.

Is there any suggested max number of images per catalog?

Is it a good idea to split my library into different catalogs? (Wouldn't do that if not necessary because it takes away the possibility to search the whole library)
Can I split/merge catalogs? (i.e. move edited photos from one catalog into another)
afx
Posts: 1675
Joined: Thu Jan 12, 2012 1:38 pm
operating_system: Linux
System_Drive: N/A
32bit or 64bit: 64 Bit
Video Card: FirePro 4900
Monitor/Display Make & Model: NEC PA301w, ColorMunki
Location: München
Contact:

Re: Catalog very slow

Post by afx »

tylmaster wrote:Is there any suggested max number of images per catalog?
Yearly would be something logical.
Is it a good idea to split my library into different catalogs? (Wouldn't do that if not necessary because it takes away the possibility to search the whole library)
AS searches in all open catalogs. You are not limited to one catalog only.
Can I split/merge catalogs? (i.e. move edited photos from one catalog into another)
Not really.
You need to re-import.

cheers
afx
Send bugs to the Monkey // AfterShot Kickstart Guide // sRGB clipping sucks and Adobe RGB is just as bad
Bibble since 2005 // W7 64 on quad Phenom // Ubuntu 14.4 on quad i7 and dualcore AMD // Images
tylmaster
Posts: 7
Joined: Mon Aug 12, 2013 2:08 pm
operating_system: Linux
System_Drive: N/A
32bit or 64bit: 64 Bit

Re: Catalog very slow

Post by tylmaster »

Thanks for the answer!
Yearly would be something logical.
Sounds quite reasonable, but would be hard to realize due to my file structure. (folders are ordered thematically)

Still: Is there a common known limitation (or suggestion) regarding the number of files?
AS searches in all open catalogs. You are not limited to one catalog only.
That's a great piece of info - maybe I'll rethink my organization...
Not really.
You need to re-import.
That's a pity. As I'm new to ASP there may be greater changes in my catalog philosophy for the next few weeks/months. Is meta information (starring, selections, color labels, keywords) saved independent of the catalog or will it be lost when I re-import the images.
Anyone knows whether this will change in future versions?

@afx: As you seem to be german, is there also a german version of your ASP survival guide?
afx
Posts: 1675
Joined: Thu Jan 12, 2012 1:38 pm
operating_system: Linux
System_Drive: N/A
32bit or 64bit: 64 Bit
Video Card: FirePro 4900
Monitor/Display Make & Model: NEC PA301w, ColorMunki
Location: München
Contact:

Re: Catalog very slow

Post by afx »

tylmaster wrote:
Yearly would be something logical.
Sounds quite reasonable, but would be hard to realize due to my file structure. (folders are ordered thematically)
Ouch, that is what metadata is for me thinks....
Still: Is there a common known limitation (or suggestion) regarding the number of files?
None that I am aware of.
It really depends on contents and available resources.
Putting the catalog on an SSD and having enough memory is helpful.
That's a pity. As I'm new to ASP there may be greater changes in my catalog philosophy for the next few weeks/months. Is meta information (starring, selections, color labels, keywords) saved independent of the catalog or will it be lost when I re-import the images.
There is an option in the preferences to write all meta info also in XMP files, so nothing is lost when the catalog gets lost.
I consider this option mandatory.
Anyone knows whether this will change in future versions?
Maybe long term, but not in the foreseeable future.
@afx: As you seem to be german, is there also a german version of your ASP survival guide?
Nö, da is der Markt leider zu klein für den Aufwand ;-(

cheers
afx
Send bugs to the Monkey // AfterShot Kickstart Guide // sRGB clipping sucks and Adobe RGB is just as bad
Bibble since 2005 // W7 64 on quad Phenom // Ubuntu 14.4 on quad i7 and dualcore AMD // Images
tylmaster
Posts: 7
Joined: Mon Aug 12, 2013 2:08 pm
operating_system: Linux
System_Drive: N/A
32bit or 64bit: 64 Bit

Re: Catalog very slow

Post by tylmaster »

afx wrote:Ouch, that is what metadata is for me thinks....
You're right, but until I found ASP some days ago I didn't know of any software that would let me work with my linux & windows machines with the same database. So I just kept the filesystem well ordered, that's something every OS and software understands.
Putting the catalog on an SSD and having enough memory is helpful.
OK, maybe my USB 2 HDD is a little bit slow for that task :?
But I think I'll start with smaller catalogs first. But I have no idea how to organize all this data. Maybe you can help me a little bit?
My folder structure looks like this:
.../category 1/yyyy-mm-dd event 1
.../category 1/yyyy-mm-dd event 2
.../category 2/yyyy-mm-dd event x
and so on. It's 12 categories with about 5-100 events inside, mostly about 50. Biggest category with >50'000 images is my private holidays folder. I also could reorder my filesystem, but I don't like this idea because it would make it difficult to find things on PCs without ASP installed.
In your kickstart guide you recommend to first cull and tag and then import new photos to the library. Would you also suggest to do so with my existing collection?
There is an option in the preferences to write all meta info also in XMP files, so nothing is lost when the catalog gets lost.
I consider this option mandatory.
One more nice piece of info, thanks.

One more question:
- thumbs are stored as images in the catalog folder
- xmp stores my changes, and metadata as soon as I set the option (all the xmp files in my photo folder are rather annoying, but I think it just has to be like that)
- what is stored in the catalog database? It already takes about 5GB which is pretty much for those few images I edited yet.
@afx: As you seem to be german, is there also a german version of your ASP survival guide?
Nö, da is der Markt leider zu klein für den Aufwand ;-(
Schade :cry: But I bought it anyway today, hope it helps (by the way: thanks for not DRMing it, that's one more reason to think my 29chf are well spent money)
afx
Posts: 1675
Joined: Thu Jan 12, 2012 1:38 pm
operating_system: Linux
System_Drive: N/A
32bit or 64bit: 64 Bit
Video Card: FirePro 4900
Monitor/Display Make & Model: NEC PA301w, ColorMunki
Location: München
Contact:

Re: Catalog very slow

Post by afx »

tylmaster wrote:My folder structure looks like this:
.../category 1/yyyy-mm-dd event 1
.../category 1/yyyy-mm-dd event 2
.../category 2/yyyy-mm-dd event x
and so on. It's 12 categories with about 5-100 events inside, mostly about 50. Biggest category with >50'000 images is my private holidays folder. I also could reorder my filesystem, but I don't like this idea because it would make it difficult to find things on PCs without ASP installed.
You could still set up catalogs per year. Of course, the import is a bit of a PITA then ;-)
But while 50K in a catalog should work, it is not really something to look forward to.
Cataloging is still the weakest part of AS ;-(
In your kickstart guide you recommend to first cull and tag and then import new photos to the library. Would you also suggest to do so with my existing collection?
Why would your existing collection still need culling?
- thumbs are stored as images in the catalog folder
- xmp stores my changes, and metadata as soon as I set the option (all the xmp files in my photo folder are rather annoying, but I think it just has to be like that)
- what is stored in the catalog database? It already takes about 5GB which is pretty much for those few images I edited yet.
Previews.
In file system mode they are stored in the cache, in catalog mode they are store in the catalog.
But I bought it anyway today, hope it helps (by the way: thanks for not DRMing it, that's one more reason to think my 29chf are well spent money)
Thanks!
Marco and I think DRM is evil and only hurts those who buy stuff legitimately.

cheers
afx
Send bugs to the Monkey // AfterShot Kickstart Guide // sRGB clipping sucks and Adobe RGB is just as bad
Bibble since 2005 // W7 64 on quad Phenom // Ubuntu 14.4 on quad i7 and dualcore AMD // Images
tylmaster
Posts: 7
Joined: Mon Aug 12, 2013 2:08 pm
operating_system: Linux
System_Drive: N/A
32bit or 64bit: 64 Bit

Re: Catalog very slow

Post by tylmaster »

After nearly 1 month of trying ASP I finally bought it. Many things make me really happy, but the catalog still makes my brain smoke.
afx wrote:Cataloging is still the weakest part of AS ;-(
seems so :(

Biggest problem by now: What happens to my library when I move/rename photos/folders in the filesystem? OK, rhetorical question. What happens is that the "new" photos aren't shown until reimport, and the "old" (=moved away) photos stay as ghosts in the catalog. No way to get rid of them.
After reimporting ASP detects the changes made to an image, but the history is lost. Is this a bug or a missing feature or am I doing something wrong?

I guess there will be a lot of folder movements in the near future, so it would be really really nice if the catalog could handle that! :!:
Why would your existing collection still need culling?
Don't ask :roll: I'm kind of a collector guy, having LOADS of unsorted photos -> that's my main reason for using ASP
Marco and I think DRM is evil and only hurts those who buy stuff legitimately.
That's right. As IT company we already earned a lot of money helping our clients with problems due to copy protection.
Post Reply