There are downsides.   Putting it another way - there is no upside to poor shareholder recordkeeping either.   Mistakes involving the cap table/share registry can be severe if it results in litigation.  Resolving disputes that could have been avoided in the first place is expensive.  Should the company decide to list on an exchange - or - be acquired (or another corporate action event) and the shareholder recordkeeping is out of whack - it might delay the event.   It would be difficult to move ahead with such a transaction if mistakes impact the valuation of the transaction and/or disenfranchises any current shareholder.  

Also, the longer you carry forward mistakes - the harder it becomes to fix the mistake in the future.  

I would suggest outsourcing matters such as shareholder recordkeeping to a professional transfer agent if you believe that you cannot properly focus or staff to it.  Investing in the governance, compliance and investor relations "best practices" will come in handy when your company grows.  One measure to reduce risk is to eliminate paper stock certificates and issue "book entry".  Lost stock certificates for larger shareholders cause a big problem - as an insurance policy will be required to replace it in the future.  Having a "clean" shareholder file will reduce significantly "surprises" down the road which can take away management time.  As the company grows - you will also have to factor in recordkeeping for options, restricted stock units and other executive compensation plans.   Spend your busy time growing your company and leave the ministerial work for the professionals.