In reply to dannyW:
You need to separate in your mind the company's profits and your own source of income ie. any salary or dividends you draw. Your tax position is to a large extent (though not entirely) independent of the company.
Your uk company will pay UK corporate tax at c20%on its profits as a starting point.
You will be subject to French tax on your personal income (eg dividends) if resident there but you may still suffer UK tax under some fairly new anti avoidance rules for short term non-residents. However, that's all subject to the double tax treaty.
There is also a possibility that the French authorities might want to tax the company dependent upon their rules on corporate tax residence, permanent establishments etc all again subject to the tax treaty.
If you have many links with the UK, you may also find it hard to shake off UK residence under the new statutory residence rules. These are deliberately worded to catch people spending a lot of time working in the UK and you would need to take great care over these and even then you may find that practically it is impossible to continue working in the UK without being UK tax resident. You could of course be dual resident, subject again to the tax treaty.
You will need some good local French tax advice and some decent UK tax advice too. This is a lot more complex than it may have originally sounded...
Post edited at 11:11