What is the Merge Split Column JMP AddIn?
The Merge Column utility (download AddIn) can "repair" columns in your Riffyn data table that get split into multiple columns due to process changes. For example, if you remove a property or resource from one version of a process on a particular step, and then put it back on the step, it will come out in your data table as two columns because the restored resource is given a unique identifier. Riffyn doesn't know they should be treated as the same resource.
In other cases you might split a step into two steps and move a resource or property to the new spit step. These resources / properties will also get split into two columns in your data table, even though you recognize that they are actually the same thing in the real world.
It would be nice if there was a way to mark these "split" columns as being actually the same and zip them back together in your data table into one column. It would be nicer if Riffyn could figure out this issue and "repair" the columns automatically.
That's what the Merge Split or Repeated Columns AddIn for JMP does for you. You access it here:
The add-in automatically:
- detects such split column issues,
- checks for data conflicts between the columns (e.g., when two columns look like they are split, but actually are truly two separate columns that should stay separate because they have overlapping/conflicting data), and
- automatically merges un-conflicted split columns into one column when the desired merge is unambiguous.
If the desired merge is ambiguous (meaning the AddIn can’t figure out what column should be merged with what), the AddIn will present the defective columns to the user to choose what to merge them into. (Or you can skip column the merge if you wish.)
Many flavors of columns. Another issue with split columns is that each column often has many "flavors" in the data table: lower spec limit, target, upper spec limit, value, unit, etc. Resources may even have tens of child columns associated with them. It's a real pain in the rump if you have to manually merge each flavor of these columns pair by pair. But no sweat, the AddIn will take care of it for you. It will find all “flavors” of columns of a property or resource and merge those automatically with their matching partner of the same flavor, so you don’t have to deal with them one by one.
Saving your merge configuration. At the end of all these merges (both automatic + user-specified), you may wish to save your merge configuration to the process so that future data exports will be automatically merged in the same way without further instruction from you. The AddIn lets you do that too. At the end of the workflow, it saves all your merge choices into a text configuration object (in JSON format) which you can paste into the process description. Every time you download that data table in the future via the Riffyn API, the Merge Columns utility (and the Get Experiment Data function) will check for such a configuration. If found, it will automatically merge all those same columns without prompting you again. You can always update/delete/replace the merge config too at any time.
How do I use the Merge Columns utility?
There are two ways to use it: automatic mode and interactive mode (which is a bit more powerful).
Automatic merge will happen when you download any experiment or process data using the Get and Join Experiment Data function in the AddIn. If you select the check box option to merge columns (see below), then last step of the data fetching method will apply the Merge Split or Repeated Column utility automatically without any user input.
This will apply any saved configurations, and merge any new repeated columns that are detected. But it will not ask for any user input. So if ambiguities are found, it will skip those columns silently and leave them split. (You can apply the merge utility again even on this resulting data table if you wish, and resolve ambiguities.) The resulting table will be titled as having merged columns:
Interactive merge will happen when you choose the Merge Split or Repeated Column menu item from the Riffyn JMP AddIn as shown above.
- First open any Riffyn data table in JMP.
- Then apply the utility.
It will first check for a merge configuration and apply it, then search for additional repeated resources and properties, then search for moved resources and properties and ask you to designate where the resource or property moved to like so:
At the end of interactive mode, it will present a JSON object to you with the merge configuration. You can copy this selected text and paste it into the process description of the process for which you just opened the data table.
And you paste this text here in the side bar:
or here in the process browser modal.
The add in works both with data downloaded directly from Riffyn or fetched via the API using the JMP AddIn. And it's smart enough to compensate for column names that do or do not include the process name.