Discovery Assistant CSV to IDX

ImageMAKER Discovery Assistant tool to convert a CSV file to an Introspect 4.5 load file format.

Zantaz IDX File Format from Introspect technical spec: www.Zantaz.com

Basic operation:

Takes a user defined set of field mappings (csv2idx.dat) and converts the contents of a CSV or Concordance DAT file to an IDX file that can be loaded into Zantaz Introspect.

CSV must contain the following fields: BEGINDOC and FILEEXTENSION.

User Interface:

Load:

• name source CSV file

User Prompts:

• BatchName
• IndexID
• FolderID
• Custotian
• Archive
• export type:

  TIFF, TIFF+TEXT, SOURCE, SOURCE+TEXT

Questions and Answers:

Q When converting from csv2idx, do you have to generate the .dat file manually or is there a way of getting this mapping via a more precise or automated method? I have the csv in the format I want with all the columns and I'm not quite sure how to proceed with the mapping.

A: The DAT file is the whole brains of the operation.

The csv2idx.dat file maps our standard CSV field names to what we believe to be standard Introspec fields names.

Introspec IDX field names are somewhat random, in that they in turn map to numeric values which I believe to be frozen in stone, but completely un-intelligible to the user. (text -> IFD mappings are listed at the start of the IDX file).

Discovery Assistant Export Defaults (requirements):

Directory Structure:

Flat export (all files in one directory).

Export Type:

Character Separate Text.

Bates Stamp OFF

Text Output ON

Source Output ON only if you are exporting Source.

Dates are American formatted M/dd/yyyy (computer region setting is USA)

Output File Naming Scheme:

User names the files to include FileID in the string.

ie: %ProjectID%%FileID%

FileID is the default index value
Bates numbers are assigned recursively
records are sorted by 'bates number'

Spec requires the following fields:

• Batch Name
• Compound ID
• Index ID
• Collection or Folder ID
• File Format
• File Path
• File Name
• Priority
• Index Flag
• Render Flag
• Hold Flag
• ... user defined fields... DocumentID.

Export Example:

1, 2,3,4, 5, 6, 7, 8,9, 10, 11, 12

"",0,0,7099,"TIF","","STRIF_Joehl007.0021.tif",2,1, 0, 0, "STRIF_Joehl007.0021"

User Defined Field mappings:

csv2idx.DAT contains a list of field mappings.

Each field mapping record contains 5 fields:

Original Name

• oringinal name in CSV file.

Mapped Name

• name to appear in IDX file.

Mapped UDF#

• assigned UDF number in header section of IDX file.

Data

• user defined data field. Can contain a constant string, or mapped fields.

Flags

• special instructions for how to handle field
  IGNORE
  • A field with this particular flag would not be outputted to the .idx file.
  DateTime_FIELD
  • The data of this field would be automatically converted to a MMDDYYY format.
  ADDRESS_FIELD
  • ‘;’ within the data would be replaced with a ‘|’ character

Example records:

DOCTITLE,Title,55004,[FROM] to [SENTTO], data field will contain resolved [FROM] and [SENTTO] values. eg: author@book.com to publisher@book.com

FCREATDATE,File Create Date,55005,,DateTime_FIELD

Application Details:

Read in first line of CSV file and determine the #fields, and field header names

<HEADER>

Status=0"

"ExternalID=1"

"DocumentTextFile" (optional)

"NumCols=x"

Unique Fields (one for each value in CSV)

Custodian field

Archive Field

<\HEADER>

Write out

BatchName (user prompt)

Compound ID=0

IndexID (user prompt)

FolderID (user prompt)

FieldName (BatesNumber)

File_Format=TIF (or source EXT otherwise)

File Path ("")

FileName name of TIFF file/ source file.

Priority 2 (normal)

IndexFlag 0 (do not create index)

RenderFlag 0 (do not place doc in the queue)

HoldFlag 0 (do not hold the document)

<user defined fields> [be sure to remove TIF and TXT]

Custodian Field (user prompt)M

DocTextFile Name of TXT file (optional)

Sample TIFF export (with comment field):

<HEADER>
"Status=0"
"NumCols=2"
"UDF1=1183","Begin Doc ID"
"UDF2=11013","Comment Field"
<\HEADER>
" ",1,0,0,"TIF","","CHN000040001.TIF",2,0,0,0,"CHN000040001""~FILE:CHN000040001_11013.LTF.txt"
" ",1,0,0,"TIF","","CHN000050001.TIF",2,0,0,0,"CHN000050001""~FILE:CHN000050001_11013.LTF.txt"
" ",1,0,0,"TIF","","CHN000060001.TIF",2,0,0,0,"CHN000060001""~FILE:CHN000060001_11013.LTF.txt"
" ",0,0,0,"TIF","","CHN000070001.TIF",2,0,0,0,"CHN000070001""~FILE:CHN000070001_11013.LTF.txt"