Solution 1: Let \(P\) and \(Q\) be the centres of the smaller and larger circles, respectively, and let \(C\) and \(D\) be the points of tangency of the smaller and larger circles to \(OB\). Similarly, let \(E\) and \(F\) be the points of tangency of the smaller and larger circles to \(OA\).
Line segments \(PC\) and \(PE\) are radii of the smaller circle and thus are equal. Line segments \(OE\) and \(OC\) are equal because the distances from two points of tangency to the point where the tangents intersect are equal. We also have that \(\angle OEP=\angle OCP=90\degree\) because a radius drawn to a point of tangency is perpendicular to that tangent. Therefore, \(\triangle OCP\) is congruent to \(\triangle OEP\) by side-angle-side congruence. This means \(\angle EOP=\angle COP\), so \(OP\) is the angle bisector of \(\angle EOC=\angle AOB\). By a similar argument, \(OQ\) is the angle bisector of \(\angle FOD=\angle AOB\). This tells us that \(P\) and \(Q\) both lie on the angle bisector of \(\angle AOB\). Therefore, \(OPQ\) is a line segment, and \(\angle POC=\dfrac{2\theta}{2}=\theta\).
Let \(R\) be the radius of the larger circle and let \(r\) be the radius of the smaller circle. It follows from the fact that a radius is perpendicular to the corresponding tangent that line segment \(PQ\) passes through the point where the two circles are tangent. This means \(PQ=r+R\). Also, since \(\dfrac{PC}{OP}=\sin\theta\) and \(PC=r\), we get \(OP=\dfrac{r}{\sin\theta}\). Therefore, \(OQ=\dfrac{r}{\sin\theta}+r+R\) (\(\sin\theta\neq 0\) because \(0\degree<\theta<45\degree\)). We also know \(\sin\theta=\dfrac{QD}{OQ}\) and \(QD=R\), so
\[\frac{R}{OQ}=\sin\theta=\frac{R}{\frac{r}{\sin\theta}+r+R}\] Multiplying through by the denominator of the expression on the right gives \[\begin{align*} R&=\sin\theta\left(\frac{r}{\sin\theta}+r+R\right) \\ &=r+r\sin\theta+R\sin\theta.\end{align*}\] Bringing all terms with an \(R\) to one side and factoring, we get \[R(1-\sin\theta)=r(1+\sin\theta)\] and so now we can solve for \(\dfrac{R}{r}\) to get \[\frac{R}{r}=\frac{1+\sin\theta}{1-\sin\theta}.\] This expression is defined because \(0<\theta<45\degree\), so \(\sin\theta\neq 1\).
Solution 2: Let \(P\) and \(Q\) be the centres of the smaller and larger circles respectively, and let \(C\) and \(D\) be the points of tangency of the smaller and larger circles to \(OB\). Let \(G\) be the point on \(QD\) so that \(PG\) is perpendicular to \(QD\).
As mentioned in the first solution, \(\angle PCD\) and \(\angle QDC\) are both right angles and \(PQ\) passes through the point at which the two circles are tangent. As well, \(O\), \(P\), and \(Q\) lie on the angle bisector of \(\angle AOB\).
This means \(PG\) is parallel to \(OD\) so \(\angle QPG=\angle QOD\). We also have \(\angle QGP=\angle QDO=90\degree\), so \(\triangle PQG\) is similar to \(\triangle OQD\). Therefore, since \(Q\) lies on the angle bisector of \(\angle AOB\), we have \(\dfrac{QG}{PQ}=\dfrac{QD}{OQ}=\sin\theta\).
Let \(R\) be the radius of the larger circle and \(r\) be the radius of the smaller circle. Since quadrilateral \(PGDC\) has three right angles, it is a rectangle, which means \(GD=PC=r\). Thus, \(QG=R-r\). We also have that \(PQ=R+r\), so \[\sin\theta=\frac{R-r}{R+r}.\] This can be rearranged to get \(R\sin\theta+r\sin\theta=R-r\) or \(R(1-\sin\theta)=r(1+\sin\theta)\), and therefore \[\frac{R}{r}=\frac{1+\sin\theta}{1-\sin\theta}.\]
We will label the triangle \(\triangle OAB\). Let \(P\) be the centre of the largest circle in the corner closest to vertex \(O\) and let \(Q\) be the centre of the largest circle (the circle inscribed in the triangle). Let the circles with centres \(P\) and \(Q\) be tangent at \(T\), and suppose the common tangent intersects \(OA\) at \(S\) and \(OB\) at \(R\). Finally, let \(D\) be the point at which the circle centred at \(Q\) is tangent to \(OB\).
By the reasoning in Question 1, points \(O\), \(P\), and \(Q\) all lie on the angle bisector of \(\angle AOB\), so \(\angle SOT=\angle ROT\). We also have, by circle properties, that \(\angle STO=\angle RTO=90\degree\). Therefore, \(\triangle STO\) is congruent to \(\triangle RTO\) by angle-side-angle congruence (these two triangles share side \(OT\)). It follows that \(\angle OST=\angle ORT\) and since \(\angle SOR=60\degree\), we get that \(\triangle SOR\) is equilateral. [Note: If we only assume that \(\triangle AOB\) is isosceles, this argument still shows that \(\triangle SOR\) is similar to \(\triangle AOB\).]
Suppose the side lengths of \(\triangle AOB\) are equal to \(x\).
Since \(OQ\) is the angle bisector of \(\angle AOB\), we have that \(\angle QOD=\dfrac{60\degree}{2}=30\degree\). Since \(OB\) is tangent to the largest circle at point \(D\), we have that \(\angle ODQ=90\degree\). Therefore, \(\triangle ODQ\) is a \(30\degree\)-\(60\degree\)-\(90\degree\) triangle which implies \(\dfrac{QD}{OD}=\dfrac{1}{\sqrt{3}}\). Since \(\triangle AOB\) is equilateral, \(D\) is the midpoint of \(OB\). [The proof of this is left as an exercise. One way to show it is to connect \(Q\) to \(B\) and show that \(\triangle QOD\) is congruent to \(\triangle QBD\).] This means \(OD=\dfrac{x}{2}\) so \(QD=\dfrac{x}{2\sqrt{3}}\). We have found the radius of the largest circle in terms of the side length of the triangle.
By Question 1 with \(\theta=30\degree\), the ratio of the radius of the circle centred at \(Q\) to the radius of the circle centred at \(P\) is \[\frac{1+\sin30\degree}{1-\sin30\degree}=\frac{1+\frac{1}{2}}{1-\frac{1}{2}}=3.\] This means the radius of the circle centred at \(P\) is \(\dfrac{1}{3}\times QD=\dfrac{x}{6\sqrt{3}}\). By symmetry, the other two circles tangent to the largest circle have this same radius.
We showed earlier that \(\triangle SOR\) is equilateral. This means we can apply the argument above again to get that the radii of the three next largest circles are each \(\dfrac{1}{3}\times\dfrac{x}{6\sqrt{3}}\). We could then draw the common tangent to the circle centred at \(P\) and the next largest circle to repeat the argument. The radius will be multiplied by \(\frac{1}{3}\) each time.
Therefore, the total area of the circles is represented by the following infinite series: \[\pi\left(\frac{x}{2\sqrt{3}}\right)^2+3\pi\left(\frac{x}{6\sqrt{3}}\right)^2+3\pi\left(\frac{x}{18\sqrt{3}}\right)^2+\cdots.\] The first term in the sum is equal to the area of the largest circle. The second term is equal to the total area of the three next largest circles (those tangent to the largest circle). The third term is equal to the total area of the three next largest circles, and so on.
After some simplification, the sum above is equivalent to \[\frac{\pi x^2}{12}+\frac{3\pi x^2}{12}\left(\frac{1}{9}+\frac{1}{9^2}+\frac{1}{9^3}+\cdots\right).\]
If \(a\) and \(r\) are real numbers with \(-1<r<1\), then we can find the sum of the geometric series \(a+ar+ar^2+\cdots\) using the formula \(a+ar+ar^2+\cdots=\dfrac{a}{1-r}\). Our expressions for the total area of the circles involves a geometric series with \(a=r=\dfrac{1}{9}\), so \[\frac{1}{9}+\frac{1}{9^2}+\frac{1}{9^3}+\cdots=\frac{\frac{1}{9}}{1-\frac{1}{9}}=\frac{1}{8}.\] Therefore, the total area of the circles is \[\begin{align*} \frac{\pi x^2}{12}+\frac{3\pi x^2}{12}\times\frac{1}{8} &= \frac{8\pi x^2+3\pi x^2}{96} \\ &= \frac{11\pi x^2}{96}.\end{align*}\]
There are several ways to determine the area of \(\triangle AOB\) in terms of its side length, \(x\). One way is to use that the area of a triangle with side lengths \(a\) and \(b\) meeting at angle \(\alpha\) is \(\dfrac{1}{2}ab\sin\alpha\). Thus, \(\triangle AOB\) has area \(\dfrac{1}{2}x^2\sin 60\degree=\dfrac{\sqrt{3}x^2}{4}\) since each of its angles measures \(60\degree\) and its sides all have length \(x\). The fraction of the triangle that is covered by circles is \[\frac{\frac{11\pi x^2}{96}}{\frac{\sqrt{3}x^2}{4}}=\frac{11\pi}{24\sqrt{3}}.\]
This calculation will be rather similar to the one in Question 2. Suppose the radius of the larger circle is \(R\) and set \(\alpha=\dfrac{1-\sin\theta}{1+\sin\theta}\). From Question 1, we have that \(\dfrac{r}{R}=\alpha\), or \(r=\alpha R\). Following the reasoning in the solution to Question 2, we can show that the total area of the circles is given by \[\pi R^2+\pi(\alpha R)^2+\pi(\alpha^2R)^2+\pi(\alpha^3 R)^2+\cdots.\] Factoring out \(\pi R^2\), this is equal to \[\pi R^2(1+\alpha^2+\alpha^4+\alpha^6+\cdots).\] Since \(0\degree<\theta<45\degree\), we have that \(0<\sin\theta<1\) (in fact, \(\sin\theta<\frac{\sqrt{2}}{2}\), but having \(\sin\theta<1\) is good enough for what follows). This means \(0<1-\sin\theta<1\). Furthermore, since \(\sin\theta\) is positive, we have \(1-\sin\theta<1+\sin\theta\). It follows that \[0<\dfrac{1-\sin\theta}{1+\sin\theta}<1\] or \(0<\alpha<1\) and so \(0<\alpha^2<1\). Therefore, using the formula for the sum of a geometric series (see Question 2), the total area of the circles is \[\pi R^2(1+\alpha^2+\alpha^4+\alpha^6+\cdots)=\frac{\pi R^2}{1-\alpha^2}.\] Substituting the expression for \(\alpha\), we have that the total area of the circles is \[\begin{align*} \frac{\pi R^2}{1-\alpha^2} &= \frac{\pi R^2}{1-\left(\frac{1-\sin\theta}{1+\sin\theta}\right)^2} \\ &= \frac{\pi R^2(1+\sin\theta)^2}{(1+\sin\theta)^2-(1-\sin\theta)^2} \\ &= \frac{\pi R^2(1+\sin\theta)^2}{4\sin\theta}.\end{align*}\]
We will return to the expression above later, but first we will find the area of \(\triangle AOB\) in terms of \(R\) and \(\theta\) in order to compute the ratio.
Let \(D\), \(F\), and \(V\) be the points of tangency of the largest circle to the three sides of \(\triangle OAB\) as shown below. Connect the centre of the circle, \(Q\), to \(O\), \(A\), \(B\), \(D\), \(F\), and \(V\).
The rest of this page is devoted to proving that \(OQV\) is a line. You may wish to skip this part of the argument and come back to it later.
Similar to an observation in Question 1, we have that \(OF=OD\) because they are equal tangents. Also, \(QF=QD=R\) and \(\triangle OQF\) and \(\triangle OQD\) have common side \(OQ\). By side-side-side congruence, \(\triangle OQF\) is congruent to \(\triangle OQD\). This means \(\angle OQF=\angle OQD\).
By similar arguments, \(\angle BQV=\angle BQD\) and \(\angle AQV=\angle AQF\).
It is given that \(OA=OB\), and since \(OF=OD\), we have \[FA=OA-OF=OB-OD=DB.\] Again, \(QF=QD=R\) and \(\angle AFQ=\angle BDQ=90\degree\) because they are each made by a tangent and a radius, so we have that \(\triangle AFQ\) is congruent to \(\triangle BDQ\) by side-angle-side congruence. This means \(\angle BQD=\angle AQF\).
Using that \(\angle OQF=\angle OQD\) and that \(\angle BQV=\angle BQD=\angle AQF=\angle AQV\), we get \[\begin{align*} 360\degree &= \angle OQD+\angle BQD+\angle BQV+\angle AQV+\angle AQF+\angle OQF \\ &= \angle OQD+\angle BQD+\angle BQV+\angle BQV+\angle BQD+\angle OQD \\ &= 2(\angle OQD+\angle BQD+\angle BQV) \\ &= 2\angle OQV\end{align*}\] This means \(\angle OQV=180\degree\), so \(OQV\) is a line segment.
By right-angle trigonometry and since the point \(Q\) lies on the angle bisector of \(\angle AOB\), \(\sin\theta=\dfrac{DQ}{OQ}=\dfrac{R}{OQ}\), so \(OQ=\dfrac{R}{\sin\theta}\). Since \(QV=R\) as well, we have that \[OV=R+\dfrac{R}{\sin\theta}.\]
We also have that \(\tan\theta=\dfrac{BV}{OV}\), which means \[BV=OV\tan\theta=\left(R+\frac{R}{\sin\theta}\right)\tan\theta.\] Since \(V\) is on the angle bisector of \(\angle AOB\), we have \(\angle AOV=\angle BOV\), so \(\triangle AOV\) is congruent to \(\triangle BOV\) by side-angle-side congruence. So, \(AV=BV\) implying \(AB=2BV\). Therefore, the area of \(\triangle OAB\) is \[\begin{align*} \frac{1}{2}\times AB\times OV &= \frac{1}{2}\times 2\left(R+\frac{R}{\sin\theta}\right)\tan\theta\left(R+\frac{R}{\sin\theta}\right) \\ &= R^2\tan\theta\left(1+\frac{1}{\sin\theta}\right)^2.\end{align*}\] Recall that the total area of the circles is \[\frac{\pi R^2(1+\sin\theta)^2}{4\sin\theta},\] so the fraction of the triangle covered by circles is \[\begin{align*} \frac{\frac{\pi R^2(1+\sin\theta)^2}{4\sin\theta}}{R^2\tan\theta\left(1+\frac{1}{\sin\theta}\right)^2} &= \frac{\pi(1+\sin\theta)^2}{4\sin\theta\tan\theta\left(1+\frac{1}{\sin\theta}\right)^2} \\ &= \frac{\pi(1+\sin\theta)^2}{4\frac{\sin^2\theta}{\cos\theta}\left(1+\frac{1}{\sin\theta}\right)^2} \\ &= \frac{\pi\cos\theta(1+\sin\theta)^2}{4\left[\sin\theta\left(1+\frac{1}{\sin\theta}\right)\right]^2} \\ &= \frac{\pi\cos\theta(1+\sin\theta)^2}{4(1+\sin\theta)^2} \\ &= \frac{\pi}{4}\cos\theta.\end{align*}\]
As mentioned in the statement of the problem, this result can be used to produce the answer from Question 2. We show how to do this now, noting that this isn’t necessarily a better way to solve Question 2.
Suppose the side length of the equilateral triangle is \(x\). We computed in Question 2 that the area of the triangle is \(\dfrac{\sqrt{3}x^2}{4}\). As well, the area of the largest circle is \(\dfrac{\pi x^2}{12}\).
In Question 2, there are three infinite "lines" of circles, each starting with the largest circle and extending toward a vertex of the triangle. By Question 3, each of these three lines of circles covers the fraction \(\dfrac{\pi}{4}\cos\theta\) of the area of the triangle, where \(\theta=\dfrac{60\degree}{2}=30\degree\). Therefore, the area of each of the three infinite lines of circles is \[\begin{align*} \frac{\pi}{4}\cos30\degree\times\frac{\sqrt{3}x^2}{4} &= \frac{\pi}{4}\times\frac{\sqrt{3}}{2}\times\frac{\sqrt{3}x^2}{4} \\ &= \frac{3\pi x^2}{32}.\end{align*}\] If we take three times this quantity, we will have computed the total area of all circles in the picture from Question 2 but will have counted the area of the largest circle three times rather than once. Therefore, the total area of the circles is \[\begin{align*} 3\times\frac{3\pi x^2}{32}-2\times\frac{\pi x^2}{12} &= \pi x^2\left(\frac{9}{32}-\frac{1}{6}\right) \\ &= \pi x^2\left(\frac{27}{96}-\frac{16}{96}\right) \\ &= \frac{11\pi x^2}{96}.\end{align*}\] Therefore, the fraction of the triangle covered by circles is \[\frac{\frac{11\pi x^2}{96}}{\frac{\sqrt{3}x^2}{4}}=\frac{11\pi}{24\sqrt{3}}\] which is indeed the answer from Question 2.