Not all dogs figure out the leash thing that quickly. He could be nervous about going outside, or he was raised in a place where he was allowed to go potty wherever he wanted. Training him to do it on a leash is key to making sure he's properly housebroken.
Crate your dog to prevent him from eliminating inside. Dogs are programmed not to go potty where they rest, so make the crate just big enough for him to stand up and turn around in.
Let him out of the crate and hook him up to his leash for walking time. The time your dog spends in the crate depends on his size and breed, but after a while, he'll likely be holding it in and ready to do his business.
Walk your dog around the neighborhood. Remain calm throughout the walk, because if you seem anxious or impatient, it could stress your dog into not going potty. If you stay calm and collected, even if your dog is slightly averse to going potty on a leash, odds are, he won't be able to resist marking some territory and generally relieving himself.
Come home after 15 minutes at the earliest. If he hasn't eliminated yet, take him back inside but keep him on the leash. Then, take him someplace safe, like a backyard, still keeping him on the leash. Ideally, this area should be distraction-free -- no kids, unfamiliar dogs, traffic or anything else like that.
Continue taking him out at routine times throughout the day. Dogs learn from repetition, consistency and positive reinforcement, so establish usual walking times. For example, you may take him out first thing in the morning, 30 minutes after meals and several other times throughout the day. Praise your dog every time he eliminates outside, but do so in a calm way, not with lavish, excited praise that can unnerve your pooch.