After water is sufficiently heated up, water flow out of the overflow pipe for about 2-3 minutes. Is it normal?

Without knowing the make and model of the water heater, the overflow pipe would be assumed to be a pressure relief valve. There should be no water coming out of this unless the pressure in the system is too high, the valve is bad or it has been activated and wont seal.

Deepens how old it is could scaled up inside or the pipe in side which allows the cold water to flow to the bottom of the tank has worn away thus getting mixing at top of tank hence lack of hot water . may need to replace cold water inlet feed pipe in tank.

There is an outstanding chance that the shower causing the problem has a required flow restrictor installed in it. If you take the shower head off and look in the connection are you should see a colored screen or similar plastic piece. Take it out. The water heater has to have a flow of at least 1/2 gallon per minute.

Indicates low water flow. possibly bad electrical connection to flow detector in dhw pipe or seized dector in pipe or blocked up heat exchanger. certainly a blockage somewhere in the pipes

http://waterheatertimer.org/Leaking-water-heater.html
http://waterheatertimer.org/How-to-replace-TP-valve.html
http://waterheatertimer.org/Repair-water-heater-valve.html

Gene

If you need further help, I’m available over the phone at https://www.6ya.com/expert/gene_9f0ef4df2f9897e7

Heating water can peculate chemicals out of water, such as metals and organics.
Especially above 138 degrees.
Have well water tested.
Turn down thermostat temperature.
Or reasonably harmless well-water bacteria can react with anode rod, or other reaction in softener, to produce colored water, and smells.
Different water can produce different sludges, brown, red, gray, black, yellow.
The sludge can also combine with deteriorating pipe flakes to clog water flow.
Sludge can accumulate over years in low spots in pipes, or at elbows and constrictions.
Have plumber flush pipes.
Or run new pipes.
Also delime tankless yearly, and flush with white vinegar as shown in product manual, and treat bacteria with H2O2 hydrogen peroxide.
http://waterheatertimer.org/pdf/Bosch_Descaling-heat-exchanger.pdf

If you need further help, I’m available over the phone at https://www.6ya.com/expert/gene_9f0ef4df2f9897e7

1) Guesses: Sounds like a bad single-handled faucet cartridge, or low water flow at faucet, or clogged water filter.
Replace faucet cartridge, clean the aerator, and/or clean sediment out of pipes below sink, and clean water filter at tankless unit each month.

2) For exact measure of problem:
Typical faucet can have 2 gallon per minute flow rate.
2 gpm flow rate means faucet fills 1 gallon container in 30 seconds. Test faucet using 1 gallon container and a cooking timer to get flow rate.
Clean aerator to improve flow rate. Compare kitchen flow rate with other faucets in home.
http://waterheatertimer.org/Low-hot-water-pressure.html

Most homes have 1/2" diameter pipe running to kitchen faucet.
100 feet of 1/2" diameter pipe holds 1.2 gallons of water. This assumes no sediment build-up inside pipes.
http://waterheatertimer.org/Water-heater-recirculation-system.html

Assuming only hot is turned on: If faucet is 100 feet away from water heater, and pipe diameter is unobstructed 1/2" diameter, and faucet has 2 gallon per minute water flow, then hot water should arrive in 36 seconds.
50 feet away, and hot water should arrive in 18 seconds.

If kitchen has single handled faucet, and the handle is raised in the middle, then equal amounts of hot and cold water are drawn at same time. This doubles amount of time for hot water to arrive.

If single handle faucet has a bad cartridge, then faucet can have crossover.
Crossover means that cold water can enter the faucet even if faucet is turned to hot.
Bad single-handled faucet cartridge can delay or stop the arrival of hot water.
http://waterheatertimer.org/Crossover.html

Tankless require maintenance including cleaning combustion parts yearly, de-lime heat exchanger frequently, and clean water filter each month.
http://waterheatertimer.org/pdf/Tankless-maintenance.pdf
http://waterheatertimer.org/pdf/Bosch_Overcoming_fluctuating_temps.pdf
http://waterheatertimer.org/Troubleshoot-Rheem-Tankless-water-heater.html

Add a comment for more help.

Take a moment and rate answer for accuracy:
And take advantage of fixya expert assistance live.
For a price, expert works with you while you work on water heater or any do-it-yourself project.
Fixya is always less expensive than a service call.

Check temperature off the water for proper functioning thermostat. Look at set points on heater's thermostat. It's made to let water out when water get's to a certain high temp. More than likely it will be the pressure relief valve going bad and not holding.

T&P releif valve can be replaced on heater. Turn water off and remove with a pipe wrench. It will come out and any hardware store will carry it. T & P pressure relief valve is 3/4 threads.

When installing new one use teflon tape on threads.

That overflow is the Pressure Relief Valve.usually they are set at 3-3.5 bar but over time they can be gunged up inside and leak/allow water to flow out. Dismantle & clean or replace. Turn off mains water and ch pipes before removing - operate valve remove & allow for some water to leak from it

replace pressure release valve