My first attempt would be to define it as:
Matter that is arranged in such a way that it generally speaking tries to maintain its shape and volume.
This would rule out rubber and any material that is elastic. We can’t say it has to be structured since amorphous materials exist. But this definition does already rule out gases and liquids.
To make up for this issue we can add a additional rule:
Solids are internally connected by forces restricting it from immediately “falling apart” (if you toss a bucket filled with a liquid you can expect it to split up while if you toss a bucket with a wooden plank in it the plank won’t spontaneously fall apart). In other words, a solid consists of matter bound together.