• Progressive shear failure of rock joints transitions from climbing wear to brittle rupture with increasing joint undulation, reducing irreversible displacement.
• Pre-peak elastic energy density correlates linearly with input energy density and shear stress squared, enabling energy-based prediction.
• Post-peak elastic energy release rate and self-sustaining instability coefficient increase with joint undulation, indicating higher brittleness.
• A new dimensionless brittleness index (BI) effectively quantifies energy balance and disaster proneness, with the most undulated joint (R4) showing the highest BI of 0.697.