Look at the shirt that you're wearing. What are they made of?
Regardless what the material is, the term for the smallest component of the fabric is the same.
It is fibre. Fibre is like the atom that construct all matter. All apparel starts from the fibre.
Almost all fabrics are made from yarns and all yarns are made from fibres. Yarn is a continuous strand of twisted threads of textile fibres, filaments or other materials which are used in weaving, knitting or other intertwining to form a textile fabric.
Staple fibre refers to short fibre. Eg: cotton, wool, rayon
Filament fibre refers to long continuous fibre. Eg: silk, rayon, manufactured fibres