Sri Chinmoy: He literally saw it as that colour. At first he would not even say what he saw. But even if he said the object was a different colour, when Sri Krishna declared a particular colour, Arjuna would immediately see that that was the real colour, and he had made a mistake. At first, if Sri Krishna asked him, he might say one colour, or he would say, "Please tell me what colour it is." He would beg Sri Krishna to say what colour it was. Then Sri Krishna would tell him, and immediately he would see it was that colour. His physical eyes would see it that way.
In the beginning if he said another colour, and then Sri Krishna said something totally different, Arjuna would look at it and he would see exactly the same colour that Sri Krishna saw. His oneness with Sri Krishna was such that he would always see the same colour that Sri Krishna saw. In the case of others, there was no oneness, so whatever colour they saw at first was fixed in their mind. They declared, "That is the colour, that is the colour."
Very nice question!