EDL 3.3 Best Practices
Suggested best practices for Enterprise Data Loader 3.3
Print Friendly View
written: 12/01/2015
last modified: 04/10/2024

This article covers some of the best practices when setting up EDL for the first time on your network.

#1 - RUN AS ADMIN

Because the EDL client talks directly to the EDL Load Service, Windows requires you to run the client as admin. This can be done by right-clicking the icon on your desktop and choosing Run As Admin. For a more permanent solution, right-click on the icon and choose Properties. On the Shortcut tab, click Advanced, check the box to Run as administrator and click OK all the way back out.

Main Benefit:
The EDL client will run correctly without causing issues due to insufficient user permissions


#2 - USE EDL UNIQUE DESCRIPTION

Whether you use it globally (Tools > Options > General) or at the event level in the event wizard, it is always recommended that you tag your part files with an EDL description.

This field was originally added in order to help identify parts from different plants, it can also be used at the event level. For instance, maybe you have different departments or groups within your organization that have different sets of QCC files and you want to keep them organized. The EDL description is a great way to do that. When you create your load event, simply specify the override EDL description on the Overrides screen (5th screen) in the wizard. All QCC files that are loaded will be tagged. Even if you have existing events, adding the EDL description will automatically update the already existing files in the database so it's never too late to fix it.

Main Benefits:
a. If you have multiple plants that make the same part number, it's an easy way to tell the part files apart.
b. EDL description can be used in all dimension filtering queries in ERS.
c. EDL description can be used for dynamically assigning permissions to part files in ERS.
d. EDL description can be used for dynamically choosing files in both export and archive events in EDL.


#3 - SET UP CAUSES AND ACTIONS

If you are using assignable cause and corrective action files in QC-CALC 3.x, make sure you add the same ones to EDL. If you only have one set of AC/CA files, you can add them globally in Tools > Options > Causes & Actions. If you have them at the administrative group level, you can also set them in the load event on the Overrides screen (5th screen) in the wizard.

Main Benefits:
The assignable causes and corrective actions you assign are automatically reflected in the database and are available for reporting in ERS including the Pareto by AC chart.


#4 - PERFORMANCE SETTINGS - CHECK DIM SOURCES AND EXTRA INFO FIELDS

Checking the first two checkboxes at the top of the Performance screen will ensure any dimension level fields are transmitted to the database for use in ERS. This comes at a slight performance cost since the fields are stored in the QCN file (outside the QCC) in QC-CALC 3.x. However, if the loads are happening quickly enough, the time will never be noticed and the benefits are great.

Main Benefits:
Dimension filters in ERS allow filtering for either the dim source or extra info field which opens lots of options. For instance, adding "*" characters to the critical dimensions in QC-CALC sets up a scenario where dimension filters can be created in ERS to automatically report on just the critical dimensions. When combined with the EDL description (above), dimension filters to get the criticals at a particular plant or department can be quickly created.

Main Cost:
Slight performance hit per QCC file since the fields are stored in the QCN.

#5 - PERFORMANCE SETTINGS - RECORD GOVERNOR

Often a customer will purchase EDL after having run QC-CALC for several years. At that point, there are potentially a lot of records to be loaded (sometimes 100,000's). To help control the data flow, a governor was added to only grab x records per round. By default this number is 1000 (see picture above), but this can be increased depending on your needs. In addition, the "Continue loading until files are complete" checkbox can help get the data in quicker than without it checked. For example, let's say a qcc file has 50,000 records in it and a customer sets up an EDL load event to run once per day at 2am. Based on the query governor at 1000 records, it will take 50 days to get all of the data into the database since it will only pull 1000 records per run. If the "Continue" checkbox is checked, it will continue grabbing the next 1000 records until the file is completely caught up.

Main Benefits: Gets it done quicker.

Main Cost: Not really any although smaller QCC files may get stuck behind larger files if they have to load many chunks of 1000 records.

#6 - EXPRESS LOADING (ONLY IF NEEDED)

Turning on this option (see picture above) will make loading larger lists of files faster. Normally, EDL checks each QCC file for new records since the last load. It then checks for changes to the EDL description, changes to the dimension fields, and edits to the records of the file. If express loading is turned on, it only checks if there are new records. If there are new records then the rest of the changes are loaded to the database at that moment. If there are no new records, none of the other changes are checked. The theory is if a QCC file is not currently active (no new records are arriving) then it's likely that it's not receiving updates either and should be skipped unless new records arrive.

Main Benefits: It can process larger lists of QCC files a lot faster than normal.

Main Cost: If a file is not receiving new records, updates that occur to the file are not loaded to the database until much later than when they occurred.

#7 - QCC FILE MONITORING
Internally QCC files are uniquely identified by a combination of their name and an internal creation date that QC-CALC inserts into the file. Whenever a QCC file is compacted or purged, QC-CALC resets the internal date. When EDL attempts to load a QCC file to the database, it looks up the file name and creation date. If found, it loads the data under the existing file. If not found, it creates a new QCC file.

Check for compacted/purged QCC files during load: If checked, when each QCC file is loaded, the normal check is performed. Whenever a QCC file with the same name and EDL description is found with a different date, it is assumed the user compacted or purged the file in QC-CALC and that this is really just a copy of the existing file. As such, in order to prevent a lot of duplication in the file lists of ERS, EDL can automatically set the archive flag of the existing QCC file and then upload the new file. EDL can also optionally email an administrator.

Check for renamed QCC files during load: If checked, when each QCC file is loaded, the normal check is performed. Whenever a QCC file has the same exact creation date as an existing QCC file within the same EDL description, it is assumed the file was simply renamed outside of QC-CALC (in Windows Explorer). When found, the existing file in the database is set to archived and the new file is loaded. EDL can also optionally email an administrator. It is considered extremely rare that two QCC files would be created at the same moment in time so the date comparison is considered to be safe.

The checkboxes pictured below are the recommended settings for QCC File Monitoring.

Main Benefits: The lists in ERS will be more manageable and less cluttered when duplicate files are flagged as archived.

Main Costs: None. If there are any issues where EDL archives the wrong file, they can easily be fixed in the ERS Administrator.

#8 - EMAIL NOTIFICATION

This is one of the most important features to set up in EDL. Because EDL is a service that is likely running on a PC far from the user, the only way it can communicate what is going on is to reach out via email. If email notifications are not set up, EDL will continue running but will be unable to notify admins when exceptions occur.

First set up the SMTP account that will be used to send the emails. This information can be obtained from your local IT department. Just ask them for the SMTP server name, the port number to use and a user account and password that is acceptable. If user name and password is not required, simply enter "test" in both the username and password boxes. Click OK when finished so the changes are committed. Do this before updating the notifications.

Next, go to the Notifications screen and enter an email address that will receive error messages when they occur. You can also optionally add an email address to the field at the bottom so EDL emails when it is successful. Use this field only if you are running the event on a longer period, such as one per day or enjoy a full inbox.

Main Benefit: Notification when an exception occurs.

Main Costs: Extra emails if you have a lot of exceptions.

Applies To

Enterprise Data Loader v.3.3

Category

Importing Data
See more articles in this category