LoRa is a popular LPWAN communication technique operating in the unlicensed sub-GHz ISM bands. Its long range however also results in increased interference from other LoRa and non-LoRa networks, undermining network throughput due to packet collisions. This motivated extensive research in the area of collision resolution techniques for concurrent LoRa transmissions and continues to be a topic of interest. We present OpenLoRa, an open-source, unified platform to evaluate these works and extensible for future researchers to compare against existing works. In our framework, you can find implementations of four techniques CIC, FTrack, NScale, CoLoRa in addition to Standard LoRa in Python. We also present demodulator and decoder as separate blocks to provide benchmarks for future demodulators that can be plugged into this framework for fair and easy comparison against existing works. Users can also find extensive datasets of raw I/Q samples capturing LoRa node interference. Our evaluation indicates that existing contention resolution techniques fall short in their throughput performance in practical deployments, especially due to poor packet detection in low and ultra-low SNR regimes.