Ss 2.3 Botching Uploads - A Test

[dex Otaku]

Please note that I am not measuring transfer times. When possible, I am running uploads in the background with moderate-to-high system load [a/v transcoding, for example, though eMule, Apache 2.0.52, Trillian, MUSHClient, Opera (with at least 6 tabs open at all times, 3 of which refresh periodically by themselves), Outlook Express, Foobar2000, Norton AV, Windowblinds 4.x, and answering machine software are all running concurrently at all times. Among other things.].

Test #1 - uploading 595 1-minute tracks recorded at HiLP.

Result: no errors; split the transfer into several runs, of <50, 100, and the remaining tracks.

Special note: All tests passed with heavy background processes running.

Test #2: single 28-minute PCM track from MD80.

Result: no problems even with moderate-to-high system load.

Test #3: multiple 5-minute PCM tracks from recorded HiMD.

Result: 20 tracks [the last being 4:54] transferred with no errors [light system load]

Test #4: single full-length PCM track from recorded HiMD.

Result: single track successfully uploaded with moderate system load.

Special note: I don't think I will ever do this again. SS uploaded what I assume was the whole track, and then [at 96% progress] it completely bogged down the computer for a total of 14 minutes. Mouse-clicks would take up to 90 seconds to respond in any given app. SS also consumed available RAM until the system was running totally on swap [part of, but not all of, the slowdown]. For a while I thought I'd found a memory leak bug, but it recovered eventually and SS is now running normally since finishing the transfer.

Related point: my SS temporary folder is on the same physical drive as the SS library folder. This is no doubt a part of the problem, as it seemed to occur when SS was copying the <1GB file to the library. Still - copying the file should not consume CPU -or- memory like this.

As a result.. Advice: Never record your HiMD discs as huge single tracks. Yes, the transfer worked, but I've never seen SS behave that way before, even with completely-full, single-track MD80s, or near-full HiMDs made up of multiple PCM tracks.

Test #5: 96 HiSP tracks of various length making up a full HiMD.

Part 1: 16 tracks at once. No failures, light-to-moderate system load.

Part 2: 32 tracks.

Part 3: Remaining tracks, with high system load. At track 67 the whole system paused for about 1 minute; I switched to SS - it was sitting at 97% progress on the current track. After about 10 seconds it continued without further mishap. The rest of the tracks completed successfully

I'm taking this all as a good thing. If anyone has any ideas on another nice way to torture-test this.. just shout.

For now I'm working with the assumption that the bug has been taken care of. I will still back up important recordings first, out of sheer paranoia, but as I said - promising.

[Christopher]

I love you, Dex. If it succeeds, this will be a test that everyone here who uses uploading can be thankful for.

[dex Otaku]

bump

[Christopher]

Test #4 interests me. What're your system specs?

[dex Otaku]

System specs:

Nforce2-based motherboard; AMD Athlon XP 2500+ [barton core, 1.833GHz]; Samsung 512MD DDR 400 RAM; Kensington USB trackball; Panasonic DVD-RAM [-R/-RW/RAM/CDR/CDRW writer]; Optorite 40x12x40 CD-R; 2 x WD 80GB 7200RPM ATA100 hdd; separate ATA133 controller for hard discs [each IDE device has its own interface to itself]; ESS 56k voice/faxmodem; Revolution 7.1 sound card; Nvidia MV34 [FX 5200] 128MB dual-head AGP 8x video with analogue video in/out

I think that's it.

[mrsoul]

Test #4 was the problem for me as well. I no longer attempt to upload PCM as 1 large track. Luckily, most of the bands I record don't follow the Miles Davis 1970's style of 1 long track :-)

Chris and Dex-

This should go in the FAQ or pinned since most folks might not know that it is best to avoid uploading a very large PCM file. The 2 big problem uploads I had were both 1 large PCM file uploads. I have had no problems at all since I started tracking into smaller files before uploading.

[Christopher]

I'll have to give it a shot on my 2.8ghz A64 w/ 1 gig of 2-2-2-5 native DDR400..

btw, the only USB device attached is the trackball, right?

[dex Otaku]

Yes, the only USB device is the trackball.

I have other things, like a 2nd NIC for P2P, a 1394 card, a cheapo webcam, &c. but I only connect them or install them when they're actually being used.

[mrsoul]

Chris-

Even if it works on your system all that proves is that you need some serious processing and RAM power to handle that transfer for some reason. That's not exactly good news but it's worth testing...

I have an Athlon 2400 with 1GB of RAM and still had the same problems and worse.

Just making an observation.

[etotheix]

dex! This is great information. This is the kind of thing that makes this forum great.

[hanz0]

im gonna be recording a 2 hour concert in PCM. what is the best way to upload these files? i was planning on using the wav conversion tool. what is the safest way to do this, without losing any tracks?

[hanz0]

nevermind, i read the guide to uploading hi-md thread. what real risk am i at by uploading straight to sonic stage?

[dex Otaku]

ATM, I'd say the risk is very low, but then - how important are your recordings to you? Very low = non-zero.

[ROMBUSTERS]

im gonna be recording a 2 hour concert in PCM. what is the best way to upload these files? i was planning on using the wav conversion tool. what is the safest way to do this, without losing any tracks?

←

i havnt had any problems uploading or converting however be warned that uploading at PCM especially a 2-hour file would take a very long time to do

[hanz0]

thanks for doing all those tests. im sure ill be fine. btw, these recordings will be extremely important to me

::edit::

in reply to rombusters, im thinking to do it in Hi-SP, cause by doing it in PCM i would have to change discs during the show. i know by recording in hi-sp it will save me a lot of time and troubles, but is the quailty loss worth it?

Edited January 26, 2005 by hanz0

[dex Otaku]

The quality loss is very low, so much so that for most material, you won't notice it.

As long as you don't do successive generations of lossy encoding things should be fine. The greatest number of lossy passes I make with anything are two: HiSP converted to PCM for editing, PCM gets backed up, things get converted to mp3 for distribution. It's pretty much unnoticeable that way, though as always - it depends on what you're recording. Some things cause more artifacts than others.

[ROMBUSTERS]

the quality loss (especially while recording via line-in or mic) would be little to none. not to say there isnt some, its just that you probably wont hear it anyways. HiSP is a very decent codec

[hanz0]

thanks dex and rombusters. i think i will go with the hi-sp. ill play with a little til then.

thanks again

[hanz0]

in your experiances, when uploading into SS, would you recommend uploading tracks one at a time, or all at once? thanks

[dex Otaku]

in your experiances, when uploading into SS, would you recommend uploading tracks one at a time, or all at once? thanks

←

I usually do transfers in batches of about 4 tracks.

With previous versions of SS, it seemed as though the likelihood of lost tracks was higher if you ran the upload in large batches.

This is also a good reason to keep your recording split up into shorter tracks. This way if one or more are lost, you only need to pull small segments out of the backup you've hopefully made beforehand.

I still haven't had any problems with SS 2.3 uploading, though.