Merging and Reshaping Datasets: Joins, Appends and Wide-Long Conversion

What this service covers

• Merging datasets on IDs or composite keys, one-to-one and one-to-many
• Appending waves, branches or batches with consistent variables
• Reshaping between wide and long formats for panel and repeated measures
• Fuzzy matching when IDs are missing or inconsistent
• Harmonising variable names, codes and units across sources
• Checks for unmatched, duplicated and orphaned records
• Documentation of every join and its results

Why merges go wrong

A careless merge can silently duplicate records or drop them. A one-to-many join multiplies rows; mismatched IDs leave gaps. We check record counts before and after every operation, list unmatched cases and investigate why they did not match, so the final dataset has exactly the rows you expect.

Wide versus long

Repeated measurements can be stored with one row per person (wide) or one row per person per time point (long). Some analyses, such as mixed models and many charts, need long format; others need wide. We convert between them and verify that nothing was lost.

Harmonising sources

Different sources describe the same thing differently: different date formats, category codes and column names. We harmonise them, keeping a mapping table for transparency. The result feeds directly into analysis or into a data model for dashboards.

Before and after

Start with cleaning and validation so keys are reliable, then finish with coding and missing data checks. If records still need to be entered from paper, see data entry and digitisation.

How the work runs

1. You send the files and describe how they relate.
2. We profile keys and plan the merge.
3. We merge or reshape and reconcile counts.
4. You receive the combined dataset and a merge report.

What you receive

• Merged or reshaped dataset
• Merge report with counts matched and unmatched
• List of unresolved records
• Mapping tables for harmonised variables
• Reusable script