Monday, May 10, 2010

Importing EndNote Records into Zotero

Update: See Take Two for an updated Zotero RIS translator and a new version of EndNote's RIS Output Style.

It took me about 8-10 hours to import my database of 4000+ records from EndNote X1 into Zotero. I decided not to worry about my EndNote Groups, but the rest of my data (I think), and my links to PDF files, transferred successfully. This is what I did:
  1. Delete duplicate records - For some reason, EndNote created multiple copies of some of my records (another reason to switch from EndNote). I used EndNote's "Find Duplicates" function to create a subset of about 800 records, and deleted a few hundred duplicates manually.
  2. Modify EndNote's RIS Output Style (time consuming technical step) - The basic process for importing records from EndNote to Zotero is described on the Zotero website here. There are also extensive Zotero forums discussion (beginning in 2006) with additional technical information here and, to a lesser extent, here. Zotero's recommendation, which I followed, is to export data using EndNote's Reference Manager (RIS) Output Style, save as a text file, and then open the text file in Zotero. The process works smoothly, but I encountered two major issues with my own data: 
  3. a. The RIS text file did not include a Customized data field I use all the time in EndNote to indicate when I read a book or article. I was not about to lose this information or reenter it manually.
    b. The RIS text file tagged EndNote's "Label" field, which I use to indicate where books and articles are located (in the library, in my personal collection, in my filing cabinet, on my hard drive, etc.), as a note (N1). I wanted this field to appear on Zotero's "Info" column (perhaps as Archive or "Loc. in Archive" or even as "Library Catalog"), not as a separate note.
    It took me quite awhile to find a suitable workaround. I have worked enough with EndNote's import and export filters to know that just about any EndNote field can be linked to just about any field in another program or bibliography layout. It just takes time to (re)learn how it is done, and to troubleshoot what isn't working. Unfortunately, Zotero's import filters are not adjustable in the same way--unless you are a programmer, but even then changes to Zotero code would presumably have to be approved. So my options were limited. I had to find out which RIS field codes the Zotero import filter recognizes and how it transfers them. Then I had to choose fields that would work best for me. I found the necessary details here:
    •  This webpage explains RIS field code labels. This apparently does the same thing.
    •  This page identifies Zotero field codes, which helps explain the code for the RIS Zotero translator. The latter explains which RIS field codes the Zotero import filter recognizes.
    In the end, for problem a., I decided to adjust EndNote's RIS Output style to tag EndNote's "Label" field as a Keyword (KW), which Zotero reads as a tag. This means that my Location data is listed along with subject keywords in Zotero, but tags are easily searchable, and this is better than including the information as a separate note. It works.
    For problem b., I linked my "Date Read" (Custom 1) EndNote field to M1, which Zotero inserts into the Extra field in Zotero's record info column. Unfortunately, I can't sort my records by "Date Read" (the Extra field) as I can in EndNote, but at least the data is there as a field. (Zotero also doesn't list the total number of records in a library. Why is that?)
  4. Highlight All Records and Copy Formatted in EndNote - I couldn't get my revised RIS Output Style to output correctly as a text file, so I selected all records, and selected "copy formatted" in EndNote, and then pasted the whole thing into an empty WordPad file.
  5. Update Links to PDF Files - EndNote doesn't provide the full pathname to PDF files. Zotero offers a script that can make the changes for you. I found it easier to do a global search and replace with the correct path name.
  6. Save as a Unicode Text file. This is important if you have any German, Greek or Hebrew text in the file you want to import.
  7. Import into Zotero as described here.


    Ken Penner said...

    If you find duplicates in EndNote annoying, Zotero is much more so. In EndNote at least there is a mechanism for finding and eliminating duplicates. Not so with Zotero!

    d. miller said...

    As far as I can tell, the problem was not with my accidentally adding duplicates, but with EndNote creating them. I don't expect Zotero has *that* problem.