Many times what your machine learning algorithm is optimizing for is not what you really want. Worse, many times you do not really know what you want. In interactive machine learning the human is an integral part of the learning process leading to cooperative human-machine solutions. In this talk we will see a few such problems from the field of Natural Language Processing (NLP), their challenges, common pitfalls and some solutions.