Advanced User Import

** We are retiring this interface. Use at your own risk. Please contact us if you need alternatives, as they do exist. **

This feature allows the importation of a number of accounts for the access control system, and limited processing and updating of those fields as the records are imported. The operations that can be performed on each record are:

• The selection of one or more fields as matching fields to determine which existing account, if any, is to be processed;
• updating of a field, replacing the value of the field with a new value;
• updating of list fields (e.g. keyword lists), by adding, replacing, or deleting contents of the fields;
• normalizing values for a field by trimming whitespace, changing case, etc.

Files to be imported must be tab-delimited files, with the same number of fields on each line. Any characters can be present in the files, and there is no way to enter a tab character as the value of a field. The downside of this is that you can't put a tab in a field, but the advantage is that the format is so simple that any system should be able to generate it. If you cut and paste from excel into a plain text document, this is the format that results, which is also convenient.

The import process is controlled by another tab-delimited file with the same restrictions. This file contains the names of the columns in the import, the Tizra property types that should be used for each imported value, and any processing steps that should be applied to the values on import.

User Import Format

When the advanced user import format is enabled, there is a new advanced site property called defaultUserImportFormat. When this property is present, it holds a text value that will control the default settings for the user import process. This can be overridden for a particular import, but since most user import scenarios are repetitive, this removes a possible source of problems for normal production use of the import. The user import is activated by the upload button in the user administration area. When advanced imports are enabled, a dropdown in the upload dialog allows the selection of advanced import as an option. When selected the default import specification will be filled into the import field; another value can be pasted in if this import should use nonstandard specifications for some reason (such as an initial import, or some kind of special cleanup like the deletion or alteration of a set of accounts).

I strongly recommend editing import definitions in Excel and then pasting them into the administrative interface as the last step, because it's hard to see the significant alignments of the tab fields in a text editor. This is especially true for complex imports as the import codes may become long in that case. It's best to set the excel cells as "text" cells, and to turn on wrapping, so that editing won't be painful.

The import definition

The import definition is four lines (and might at some point be allowed as a header on the import file itself).

Line 1: Import type

For account imports this is one word: UserData. Additional tab-delimited fields on this line are ignored as long as they are blank; additional values may be allowed later, but ignoring blank fields makes pasting from excel simple.

Line 2: Property names

This line, and the following header lines specify the fields to be interpreted on import. The data file (and subsequent import definition lines) must have the same number of columns on every line.

The values on this line are the names of the properties for the Tizra accounts. You should not put spaces in the property names (although it is allowed), as that will make some kinds of processing codes impossible to specify, as space is the main delimiter in the processing codes.

The field name UserName is special for user accounts, and will define the user account name. It is not checked syntactically, so it does not need to be an email address, but if it is not, system-generated mail will not work for that account. Note: The value will be converted to lower case on import, which is important for checking whether the import worked properly.

Line 3: Property types

These are the same as Tizra property types (there are actually a few additional ones that we don't currently show in the user interface. The types are:

• keep preserve the existing field type for a field that already exists. Attempting to keep the type of a nonexistent field will cause an error and abort the import.
• text a long text field; Tizra 'Text' property
• html a long text field with HTML editing; Tizra 'HTML' property
• string a short text field; Tizra 'String' property
• keyword a single keyword (matched as a whole); Tizra "Single Keyword" property
• integer a whole number; Tizra "Number" property
• float a number that may include fractional values; Tizra "Number with Decimals" property
• date a date; Tizra "Date" property. The following formats are allowed: yyyy-MM-dd, yyyy/MM/dd, MM/dd/yyyy, MM-dd-yyyy, yyyyMMdd, yyyyMMddkkmmss.
• boolean a binary flag; Tizra "True/False" property
• List values keyword-list, string-list, etc. adding -list to the name of any type other than html or text specifies a property that is a list of values of the specified type. keyword-list is one of the most common types in Tizra, as in user tags and AdminTags. string-list is the type used for things like the "Authors" field on books.

As the first step of the input process, a property will be created with that name and type -- if the property already exists, the type of that property will be changed. The types of system properties can't be redefined -- attempts will be ignored.

If you only need the property temporarily, such as one to determine if the account is active or to set up licenses, you still have to define it with a type. After the import is finished you can delete all of the unnecessary fields.

Data is stored in string form internally, so data will not be lost if you change the type of a property, but subsequent editing or use of those fields may be affected if the type definition has changed. A lot of overlap is possible, so string-list is interchangeable with string except when the string may contain a ;.

Line 4: Processing codes

This is the complicated part of the import process. processing codes are a space-delimited set of strings denoting fairly simple operations to be applied to fields being imported. Each code will cause an effect to take place on a record, or a field value to be altered before import. Fields are processed in left-to-right order, in several phases.

Sometimes an import will include extra "control" or "status" columns, or a set of columns (e.g. product1 product2 product3) that will end up as one Tizra list field -- this is especially true as user imports are often concerned with updating many user tags that determine group membership and thus access control rights. All columns will be included by the system during the import -- any unwanted columns can be deleted afterwards (manually).

Processing codes: phase I (Determining matching fields)

The codes for all fields are scanned, and any fields marked id are used to match against pre-existing accounts. If 4 or fewer accounts match, one of them will be chosen for update, and the others will be deleted as duplicates. If more than 4 accounts match, that update record will be ignored.

Processing codes: phase II (Data extraction)

normalize and extract data fields from the input record, and specify how they should be used in updating its value. Fields (as always) are processed from left to right. The codes are processed from left to right as well. Codes that apply during this phase are:

• toupper translates all characters to upper case.
• tolower translates all characters to lower case.
• #value sets a fixed value for the field (e.g. #foo would set the field's value to foo). Useful if you need to specify a constant value to import in the load definition itself.
• value/value... Any code with a / in it is taken as a specification of a list of results composed of values from other fields. This means you can't use a / in any other kinds of processing code For instance a code like Conference/Session would prefix the value(s) of the Conference field to the value(s) of the Session field, and the results would be treated as the value of the field in question. $ can be used as a shortcut name for the value(s) of the current field.

The following tags are special to list properties only.

• spaced-list will convert all spaces to ; allowing subsequent phases to process this as a list value. If semicolon is needed in a list value, it can be escaped as %3B (case matters!). Ignored on other property types.
• comma-list will convert all , to ;. Ignored on other property types.
• add will cause the values of this field to be added to the current value of the property instead of replacing its value.
• remove will cause the current values of this field to be removed from the current value of the property instead of replacing its value.

Processing codes: phase III (Special operations for user accounts)

The following codes apply only to user accounts.

• add-user-tags add the values of this list field to the user tags.
• set-user-tags replace the current user tags with the value of this list field.
• delete-tag-prefix=something remove all tags from the user that start with a given string. Note that this cannot include a / as noted above. The string cannot contain spaces either (as codes are separated by spaces).
• password will set the password of the account to the value of the input. The field value will be set to a confirmation that the password was set, but the password will not be stored in plain text in the system.
• set-status=active will mark the account as active.
• set-status=inactive will mark the account as inactive.
• set-status=email-required will mark the account as pending confirmation by email round trip.
• set-status=pending will mark the account as pending activation, no confirmation required.

License codes.

Codes starting with the string license. can be used create one new license for an account, and set some of its properties. As each code can sets one aspect of the license, adding a license that is not just a copy of an existing license will take several fields.

These are the license processing codes:

• license.model This code will copy an existing license to the account, creating a new license. The field value must be the number of a license already in the system. This must be the first license code if a license is to be copied, as a new license will be created from scratch on processing any other license code if license.model has not already been processed. The other license. codes can be used to adjust the properties of a copied license if desired.
• license.start This will set the start date for a given license from the value of the field.
• license.end This will set the end date for a given license from the value of the field.
• license.controlled This will set the product controlled by the license. The value of the field must be the TizraId of an object that the license is to apply to. It can be any type of Tizra object: Collection, Book, Excerpt, Static Page, whatever.
• license.concurrency will set the number of concurrent sessions allowed by the license.
• license.match=(field name) will create a license linking to a MetaObject based on the MetaTag/field value of the MetaObject matching the column value.

Advanced processing codes

Conditional codes

It is possible to make any code apply conditionally depending on the value of any field. This function is limited to making a code apply if a field is equal to a specific value, or if a field is not equal to a specific value. If the code in question is a Phase II code, being compared may be normalized depending on the order of fields and the other codes in the import.

The conditional code is when:fieldame=value:anothertag or when:fieldame!value:anothertag.

fieldname is the name of a field, and value is a literal string. If $ is used as the fieldname the current field being processed is used. The value obviously cannot contain a /, or a space, but additionally cannot contain a : as that marks the end of the when: code prefix.

deleting a record

The code delete will cause the matching user to be deleted entirely from the system. It is most likely that this will be prefixed by a conditional code to check for a value, but could be used with a list of identifiers, for instance, to mass delete a list of users.

Note on Multiple Attempts To Import The Same File

To prevent accidental duplicate imports, the second and subsequent times you upload the same file it is ignored. For Tizra engineers developing a feature related to imports or testing imports locally, you can bypass this safeguard. To bypass it, create an text file which contains only one whitespace character (a ' ' or '\n' is fine). Then before each attempt to upload your actual tab-separated-values file, upload the single whitespace character file.

Troubleshooting

The upload result will usually appear in a new tab as a plain text message. Your web browser may not warn you or alert you if the message is updated or changed. You may have to check the tab manually.