First off I should say, never try to break the ice off a line guide with your hand. The colder it gets the more brittle your rod becomes & you could easily break the rod without exerting too much pressure. This is especially true at the small-diameter tip section.
The first course of dealing with ice in the guides is to prevent it from forming in the first place. One way to do that is lightly wax the guides before you go out in the cold. Use a carnauba wax, the kind you'd use on an automobile, without any added cleaners. Go over each line guide & be sure to buff the wax off with a soft, clean rag. Do this inside where it's warm. The wax will help to repel the water from hanging around on the guides & hopefully will keep ice from forming. In addition to this, make sure your fly line is clean. A clean line will repel water, too.
Some anglers will use chap stick or lip balm on their guides. I've never done this myself, but I've heard that it works really well. If you go this route, just make sure you get the chap stick cleaned off your guides at the end of each day. Chap stick is cheap, easily carried in your pocket, & you might even use it on your lips while fishing, too!!
I've also heard rumors of anglers who spray their rods down with a non-stick cooking spray to keep the ice away. Again, this isn't something I've ever done. So, who knows? I can't imagine that it's not messy & I wouldn't want to have to clean that olive oily-ish substance off my fly rod at the end of the day. I certainly wouldn't do this with a bamboo rod. In fact I have a personal rule that if it's cold enough for ice to form in the guides, I leave the bamboo rods at home. The cold can make breakage in any rod more probable, so why chance it with my bamboo rods?
Oh, & I should mention that there are products on the market made just for this problem & marketed to fly anglers. You could always go with one of those products if you're not the adventuresome type!
Of course, you could just remove ice the old fashioned way: dip the iced part of your rod in the stream. That will melt the ice as the water's warmer than the air temp. This is how we used to do it. However this is only a temporary fix & there's nothing like fighting a fish as your guides begin to freeze shut!!