A slip assembly for a downhole tool, such as a bridge plug, has a slip body and at least one insert unit with a base and one or more inserts. The slip body has an incline at one end that interfaces with an inclined surface of a cone. As this occurs, the slip body is pushed away from the tool's mandrel against a surrounding casing wall. The insert unit is disposed in the slip body with the base oriented at an angle relative to the incline, and with the one or more inserts extending from the base. In particular, the base can be disposed at or parallel to the incline, and the one or more inserts with less surface area than the base can extend perpendicular to the inline for the insert's distal ends to engage a surrounding wall of casing or the like.