fbpx
Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1

TOPIC: Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen"

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 02 Nov 2017 18:49 #91473

  • Hezel
  • Hezel's Avatar Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
  • Posts: 12
  • Thank you received: 0
:angry:
Just imagine you have a car that, then turning the stearing wheel to the left goes left, but in some cases the car decides that it is better to go right.
What a nightmare?
If you choose at the import option "Leave media in place" and fcpx "thinks" or "assumes" that the media files are on a camera card – that in my case is allways not the case - it is importing the media to the main fcpx folder. So you have your nice 4K lardge files eating up your discspace in a double copy.
WHY?
Does apple think it is more clever then the user?
Does apple think the user is a stupid idiot?
If I'm on set filming I use shotPutPro to copy (and check) my files to a hard drive. Meaning I import the whole folder structure from the camera (also necessary fot Sonys media browser).
And now fcpx is assuming it is a camera. The only solution is to change the folder structure of my media so fcpx is finally doing what it should do: "leave media in place". Meaning I have to go in every media folder to change the structure.

Why does Apple think it is more clever then the user? :sick:

Thomas

Please Log in or Create an account to join the conversation.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 08:00 #91480

  • Axel
  • Axel's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 553
  • Karma: 10
  • Thank you received: 72

Hezel wrote: If you choose at the import option "Leave media in place" and fcpx "thinks" or "assumes" that the media files are on a camera card – that in my case is allways not the case - it is importing the media to the main fcpx folder. So you have your nice 4K lardge files eating up your discspace in a double copy.



YOU made the copy, beforehand, not FCP. The copied card you *can* keep as a backup or delete it after import - up to you. No redundancy forced upon you.
EDIT: Besides that, there is a serious disadvantage in copying and importing cards through anything else than FCP. I now remember a "trick" someone mentioned some time ago: Set leave files in place in preferences, then drag&drop a manually copied card to an FCP X event. Bad idea! FCP needs certain file types to be wrapped as movs for best performance.

Hezel wrote: If I'm on set filming I use shotPutPro to copy (and check) my files to a hard drive. Meaning I import the whole folder structure from the camera (also necessary fot Sonys media browser).



You mean Catalyst Browse? Does everything right from the physical card. I know because I used to do this on a regular basis, checking shots abroad.



Hezel wrote: Why does Apple think it is more clever then the user? :sick:



Imho - and I mean humble! - there should be a way to access all media with FCP X without the import-dialog, which would be a lesser pain for both of us. The - german - company lesspain software made Kyno. It does just that - browses your cards (way more sophisticated than Sony's browser, although some Sony specific metadata such as picture profiles don't show). And you can copy anything from there safely without needing the folder structure later on, the metadata are copied along.

Actually, this just soothes people like us who worry about being forced to make proprietary copies of their originals, whereas this is no problem for the majority. Because it isn't a problem. One just has to accept the way FCP thinks about media management. If it allowed to leave original camera clips in place, many who deem themselves wiser than FCP would undoubtedly lose their clips, even more lose their metadata. And all would start rants what a shitty NLE FCP was.

For editing a wedding, I'm perfectly happy with the way FCP works. It's only when I know in advance that I will use other software like AAE that I organize my footage manually. I then want to get rid of wrapped XAVC asap and start by transcoding all useable snippets to ProRes right from he card.

But even then, instead of providing a manual backup, I copy the clips to the library (= don't leave files in place, as I then could). The whole idea of a neat package where everything I need is safely stored is just too convincing. But to each his own.

Please Log in or Create an account to join the conversation.

Last edit: by Axel.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 11:31 #91489

  • Hezel
  • Hezel's Avatar Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
  • Posts: 12
  • Thank you received: 0
Hello Axel,

here ist the workflow all my colleagues use:

documentary:
You are for 10 days in Africa (we did most of the countries) and you have to copy your files every day in a hotel with unstable power to a small raid, using your laptop. Sometimes you even have to copy in the bush at the back of your car camera cards since the animals didn't perform as expected and you wasted a lot of memory space.
Back in the hotel it usually take the whole night to copy with a checksum all cards to the raid. To make shure it is bit by bit correct and since no one wants to fidle around with the files under such conditions, we copy just the whole card using shot put pro for verifying and controlling the process even with power losses. The next morning you want to be shure that you have saved everything correctly.

advert:
Your customer likes to see the bubbles in the glass in a thousand ways, so you have to copy while you film some of your cards on set to the raid. Again using a checksum and a controlled process with shot put pro.

feature:
There is a lot of money envolved so you have to be save! We copy on the set again with a checksum and shot put pro all camera memory cards to two raids simoultaneously. One raid stays on the set the other one goes to the editing room.

In the editing room alle files go, again with a checksum, to the editing raid, using shot put pro.

No one wants to change any folder structure or something else with the files in this process. That would just create a big weak point in the process. So it is essential to keep the folders as they have been on the memory card.

Now we import from the editing raid in fcpx and of course choosing: "leave files in place".
And now fcpx is messing up the whole process by secretly starting to copy alle files inside the fcpx-project-folder on the hard drive.
fcpx wants to get busy for the rest of the day messing up the hard drive. (fcpx project folder on the hard drive of the MacPro while the video files are on a raid – this makes things faster)

There are such nice things like pop up windows:
"do you realy want to leave your files in a memory card folder – if it is a memory card your files will be lost forever, by using this card again" You cust click "YES" and everything would be fine.

But how arrogant must a software coder be to do it behind you back and against your chosen will?

My workaround:

I take alle files and folders outside the XDROOT folder (Sony) and put it in a date-folder. Now fcpx doesn't recognize the folder as a memory card folder and leaves the files in place.
But I have to do it with every folder – every day – every time …
And if someone is providing additional footage you have to be very careful that fcpx doesn't lable it as a memory card. Meaning messing up your project again so you have to go inside the fcpx project folder and start manually deleting files there. (happend to me quit often with external footage)

This is realy a car going left when you turn the stearing wheel right.

And thats the point you are ready to get a big hammer :evil:

Please Log in or Create an account to join the conversation.

Last edit: by Hezel.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 11:40 #91490

  • Tom Wolsky
  • Tom Wolsky's Avatar
  • Offline
  • Moderator
  • Moderator
  • Posts: 4351
  • Karma: 112
  • Thank you received: 697
The application has been working like this since 2011. It designed to create protected redundancy. The information about its operational behavior is available in the help files and all instructional tools.

Please Log in or Create an account to join the conversation.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 11:46 #91491

  • Hezel
  • Hezel's Avatar Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
  • Posts: 12
  • Thank you received: 0
There are concepts in software development related to the words:

Progress
Development
Improvement
Adujustment
Usability

Only because something has been bad in 2011 it has not to stay like this in 2017.

And pop up windows - user information - warnings have been established in software since 1970!

Please Log in or Create an account to join the conversation.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 12:59 #91497

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1366
  • Karma: 27
  • Thank you received: 284

Hezel wrote: ..."Leave media in place" and fcpx..."assumes" that the media files are on a camera card...it is importing the media to the main fcpx folder. So you have your nice 4K lardge files eating up your discspace in a double copy....If I'm on set filming I use shotPutPro to copy (and check) my files to a hard drive. Meaning I import the whole folder structure from the camera...And now fcpx is assuming it is a camera. The only solution is to change the folder structure of my media so fcpx is finally doing what it should do: "leave media in place". Meaning I have to go in every media folder to change the structure....


Hezel you are right this is a significant problem for large productions with high shooting ratios. Like you, my documentary group uses ShotPut Pro to daily offload multiple camera cards in the field -- up to 1 terabyte per day. This is checksummed and produces an offload report. We could not possibly use FCPX to offload a single camera card at a time. The data is then immediately duplicated to another on-site RAID drive for redundancy.

In post we always import the media with "leave files in place". The reasons are obvious:

(1) A "copy to library" import will duplicate the data, which we don't need -- it's already duplicated.
(2) A "copy to library" import will rename and restructure folders, making it difficult to later do verification checks. IOW it becomes much more difficult to later back track and compare filenames and folder structure.
(3) In-place import is much faster (to import)
(4) In-place import produces no slower editing performance than a "copy to library" import where files are rewrapped -- with the single exception of AVCHD which is vastly slower. This is probably a bug.

The current behavior encourages users to copy bare media files out of the folder tree to obtain an in-place import. This is a poor practice because it theoretically risks losing metadata. I haven't see that occur on XAVC-S files but it's a potential problem, and FCPX encourages this.

It appears that FCPX could selectively disallow in-place import from SD cards only, while permitting it from a hard drive copy of the same folder tree. In terminal, the SD card can be identified via "diskutil info /dev/disk2", where /dev/disk2 is (for example) the SD card reader. The returned data clearly identifies this as "Device: SD Card Reader". FCPX would simply have to do this programmatically, enumerating the connected drives and disallow "in place" only for the SD card.

If that didn't work, a popup which warns the user of a tree-based in-place import is possible. FCPX already has lots of pop-up warnings for common tasks such as deleting a media file which produces "One or more media files will be moved to the trash....External files will remain where they are". I wonder how many new FCPX users understand what that means. Since such potentially confusing messages are already there, why not add one more dialog which is actually useful and permits in-place import of tree-oriented media?

Admittedly there could be problems with this and special cases would be required, such as for AVCHD. As currently designed that must be rewrapped so it must disallow in-place import for this. That in turn requires selectively identifying AVCHD and all other formats which require rewrapping. The code identifying and implementing this internal distinction would have to be maintained and updated as new formats are released. All of that requires coding, testing, ongoing maintenance and support. That may be one reason they haven't done it.

Basically the advanced users already have in-place import of tree-based media by copying bare media files to a disk-based import folder. The users (not Apple) assume the responsibility for this. It seems to generally work very well except for AVCHD, although the missing metadata files (XML, etc) might cause problems in some cases. So we have that functionality now, and Apple does not have to code, test, maintain and support a possibly tricky area.

Please Log in or Create an account to join the conversation.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 13:18 #91498

  • Axel
  • Axel's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 553
  • Karma: 10
  • Thank you received: 72
Hello Thomas, thank you for describing your particular challenges in detail.

Hezel wrote: There is a lot of money envolved so you have to be save! We copy on the set again with a checksum and shot put pro all camera memory cards to two raids simoultaneously. One raid stays on the set the other one goes to the editing room.


You accept a lot of redundancy for safety reasons, but you are angry that FCP wants another copy.

You could launch FCP, hit cmd + i and open your first copied card. But the import window isn't nearly as mighty as the clip browser. You can't, for instance, favorite, tag or rename the clips on first viewing. So there is an understandable tendency to import all and do that later.

Am I right that the actual reason for your frustration is not so much occupied disk space but an inefficient workflow, if a lot of coverage and many hours of clips are involved?

Have a look at Kyno (1 month free trial):

It adds the - imo - missing functionality of being able to qualify, subclip and tag every clip prior to import. You need to copy as well, that's right, but you can just rewrap the selections in the background, getting rid of the unusable stuff, in some of your scenarios even on location. These files can be left in place by FCP.

Unless FCP one day allows to leave camera clips in place - which I doubt - this is as good as it gets. And if you still resent that your copies can't be used directly and if that's a major problem for you: switch to Premiere. Premiere is famous for just linking to clips without actually ingesting them. I have said a lot of bad things about Premiere, but I checked the latest version in a one-week demo and found it stable, responsive and, well, okay. At some occasion, since I last used it, they even added transcode and copy options :huh:

Please Log in or Create an account to join the conversation.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 14:43 #91499

  • Hezel
  • Hezel's Avatar Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
  • Posts: 12
  • Thank you received: 0
Hello Axel,

thank you for the Keyno tip!
But now I use:

a. Shot Put Pro to offload on set and to the editing raid
b. fcpX to edit
c. X2Pro to send data to Pro Tools for sound mixing
d. DaVinci Resolve for color grading
e. After Effects for special effects and animation
f. fcpX for final output and putting things together

meaning to fit in another software for selecting and sorting clips – a basic duty of a editing programme

Only because plain Englisch "leave files in place" means "copy behind my back anyway".

With tools you want control, that is the first thing you want!

If a chain saw goes towards your legs because it thinks, that the concrete you try to cut will ruin the chain, it is not a tool you want to deal with.

If I come home with 5TB of filme footage I don't want to watch fcpX trying to copy it in a fcpx-project-folder on my 1TB MacPro SSD Harddrive, telling my 8 hours later that I should create more disc space.

Pemiere: May be you are right, just leave – after a long learning process – fcpx to the YouTube "my sweet littel cat" video producers.

@ joema

Now Apple must keep up with different camera card formats and adujust the software every time a new file structure is coming on the market.
And thats what I call, software is in puberty, thinking it is more clever then you, but completly wrong since it is not a camera memory card but a raid the file is on.

Diskutil probably doesn't work since the raid is on a dev/disk registered.

Editing is a bit like playing the piano. Bad for you if you are a jazzer and the piano is not playing out of the major scale, so playing differnet tones then the ones you hit on the keyboard.

Loosing control thats what annoys me!

Since there is a lot of money and responsability envolved in film making.
Especially in this first and basic steps you need control and the absolut reliability what happens with your footage.

Please Log in or Create an account to join the conversation.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 16:41 #91505

  • Karsten Schlüter
  • Karsten Schlüter's Avatar
  • Offline
  • Moderator
  • Moderator
  • Posts: 3660
  • Karma: 61
  • Thank you received: 648
… there's a lot of assumptions in your rant, Hazel.

a) there's no need to keep the Lib on your size limited int. drive. On contrary: best-practice is to distribute media (and projects and cache and backups) on several drives.

b) only media which has to be re-wrapped, such as AVCHD gets 'copied'; if your source is e.g. mp4, files-in-place keeps files in place; all what FCPX do is creating symLinks - which pretend to be a copy, but don't occupy any drive space.


Please Log in or Create an account to join the conversation.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 16:49 #91506

  • Hezel
  • Hezel's Avatar Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
  • Posts: 12
  • Thank you received: 0
@ Karsten Schlüter
a. I have my files on a raid with 8 drives (Thunderboldt G-Raid), the project is on internal SSD -> this the fastest option for 4K
b. sym-links is what I would like to have but fcpx is copying behind my back, instead of linking, because it is ignoring the "keep files in place"

see posts above

Please Log in or Create an account to join the conversation.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 17:26 #91508

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1366
  • Karma: 27
  • Thank you received: 284

Karsten Schlüter wrote: … only media which has to be re-wrapped, such as AVCHD gets 'copied'....


That is really not correct -- XAVC-S does not *require* re-wrapping -- performance is good after importing in place, and I've never seen a functionality problem. But every format is different. Importing AVCHD in place causes huge I/O problems, and other formats might look OK at first but problems could appear later.

We generally assume that (1) FCPX disallows in-place import for tree-based media as a user protection to prevent loss of data for naive users who would try in-place import from an SD card, and (2) Apple does not prioritize giving experienced users an "in place" override for tree-based media.

However this may not be correct. If Apple provided in-place import from tree-based media, where would they put the metadata from the XML and other files in that tree -- without rewrapping it which merges that to a single file. They could just disregard it as users do who copy the video files to a folder for in-place import. However that is really not correct procedure -- you never know when some metadata will be needed for clip spanning, color profiles, etc. We users can get away with this in limited cases like XAVC-S because we have tested it but Apple would not sanction that (and take the risk) by coding that behavior.

The current procedure by Apple, EditReady, etc. is to merge that metadata into the video file during rewrap. But that requires rewriting the entire video file, which requires space and time. However it is technically possible to leave the tree on disk, read the metadata into some internal sidecar file or a database and use that. However that would be a lot of development and testing, and for what -- a relatively few high-end users like me and Hezel who can already avoid it by copying the media files to a folder and importing from there.

So upon further thought there may be other issues at play than Apple trying to protect naive users.

Please Log in or Create an account to join the conversation.

Last edit: by joema.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 17:28 #91509

  • Axel
  • Axel's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 553
  • Karma: 10
  • Thank you received: 72

Hezel wrote: a. I have my files on a raid with 8 drives (Thunderboldt G-Raid), the project is on internal SSD -> this the fastest option for 4K



But a second TB raid would be fast enough. Because even an internal fast SSD will considerably slow down eventually with too much space occupied. Just saying ...

Hezel wrote: b. sym-links is what I would like to have but fcpx is copying behind my back, instead of linking, because it is ignoring the "keep files in place"



If you do not have AVCHD but something else (like XAVC, I just guessed you might use an FS7), you can as well try the aforementioned trick to drag & drop your card-folder on an FCP X event. If leave files in place is checked in preferences, it should work. Give it a try.

Please Log in or Create an account to join the conversation.

Last edit: by Axel.

Import "Leave media in Place" German: "Dateien am ursprünglichen Ort lassen" 03 Nov 2017 19:53 #91511

  • Tom Wolsky
  • Tom Wolsky's Avatar
  • Offline
  • Moderator
  • Moderator
  • Posts: 4351
  • Karma: 112
  • Thank you received: 697
There is no need to store the library on the system drives ve nor to store the media inside the library. If you don’t need redundancy for safety you can store the media on the same drive as the camera archives. If the files aren’t rewrapoed or changed the system should use hard links without expending duplicate drive space.

Please Log in or Create an account to join the conversation.

  • Page:
  • 1